stdlib: add a format_text function for text-like format strings by Octachron · Pull Request #13569 · ocaml/ocaml

Octachron · 2024-10-23T14:17:36Z

When converting warning messages to Format_doc.t, it was quite annoying to add formatting hints on every space in text paragraphs, for instance:

        "This primitive declaration uses type %a,@ whose@ representation@ \
         may be either boxed or unboxed.@ Without@ an@ annotation@ to@ \
         indicate@ which@ representation@ is@ intended,@ the@ boxed@ \
         representation@ has@ been@ selected@ by@ default.@ This@ default@ \
         choice@ may@ change@ in@ future@ versions@ of@ the@ compiler,@ \
         breaking@ the@ primitive@ implementation.@ You@ should@ explicitly@ \
         annotate@ the@ declaration@ of@ %a@ with@ %a@ or@ %a,@ so@ that@ its@ \
         external@ interface@ remains@ stable@ in@ the future."

In order to remove this annoyance, this PR propose to add a format_text function which replaces spaces and newlines ([' '] and ['\n']) with formatting hints in format string literals. This means that with this new function, one can write

Format.format_text
        "This primitive declaration uses type %a, whose representation \
         may be either boxed or unboxed. Without an annotation to \
         indicate which representation is intended, the boxed \
         representation has been selected by default. This default \
         choice may change in future versions of the compiler, \
         breaking the primitive implementation. You should explicitly \
         annotate the declaration of %a with %a or %a, so that its \
         external interface remains stable in the future."

and obtain a better version of the formatted string literal above.

Compared to pp_print_string, format_text is less efficient (it may allocate a copy of the string) but it does not require to separate the text contents from the specifiers, formatting tags and other formatting hints making it easier
to use in many context.

dbuenzli · 2024-10-23T14:36:39Z

I think it would be nice to clarify and/or add a few things here:

What happens if there are multiple spaces ? Are they collapsed to a single space like in HTML/TeX etc. ?
Is there a way to avoid breaks (.e.g. U+00A0 NO BREAK SPACE, UTF-8: \xC2\xA0, or escape) ?
Is there a way to force line breaks ?
Can we make a blank line force two line breaks ? That is support a notion of paragraphs.

gasche · 2024-10-23T14:52:37Z

There could be a shorter implementation, but I don't know if it is conceptually better -- you will be disappointed that there is less exciting GADT programming. Format has a machinery to parse a format string at runtime, and check that it is compatible with a given format type -- see CamlinternalFormat.format_of_string_{format,fmtty}. So if you can transform the string representation of the format, and call these functions to parse the result back as a format of the same type.

Now that I think of it, it may actually be tricky to recognize, just in the format string, which spaces are part of the output and should turn into @ , and which are part of the non-literal format syntax. So maybe that approach does not work well.

gasche · 2024-10-23T14:57:37Z

+type neutral_concat =
+  { f:
+      'a 'b 'c 'd 'e 'f. [`String of string | `Char of char ] ->
+      ('a,'b,'c,'d,'e,'f) fmt -> ('a,'b,'c,'d,'e,'f) fmt


You could also express this as the following, if you wanted -- reusing the type of End_of_format as the generic type of parameter-less formats.

'f 'b 'c 'e. [ `String of string | `Char of char ] -> ('f, 'b, 'c, 'e, 'e, 'f) fmt

Octachron · 2024-10-23T15:09:55Z

@dbuenzli :

k contiguous spaces are translated to a @;<0 k> break hints.
only "\u{20}" is converted to break hints, other white spaces and in particular non-breaking spaces are not affected
(which are probably the other important space in text outside of CJK texts)
@\n works with its usual composability issue.
not at this point: this sounds potentially nice and useful.

@gasche , I have the impression that in term of complexity using format_of_string only avoids the straightforward helper in CamlnternalFormatBasics at the cost of a parser call and probably mishandling @[<@[h 2@]>@] at the string level.

dbuenzli · 2024-10-23T15:27:09Z

Thanks for your answers @Octachron would likely good to add them in some form to the doc string.

2. non-breaking spaces are not affected

Note: the problem is that using "\u{00A0}" (or its UTF-8 literal byte sequence) will derail the pretty printing process since it spans two bytes. So I feel something is missing to specify non breakable space (if you want to make it confusing it could be "@ " :-).

Octachron · 2024-10-23T21:13:37Z

For this point, I think it would be better to partially solve the issue at the output device level, see #13570 .

gasche · 2024-10-24T14:41:09Z

I'm satisfied with the tweaked implementation. @dbuenzli, are you satisfied with the proposed specification?

dbuenzli

I think some clarifications are needed.

dbuenzli · 2024-10-24T18:28:27Z

+    with hint breaks or forced newlines:
+- Only ascii spaces ([" " = "\u{20}"]) and linefeeds ["\n" = "\u{A}"] are
+  replaced. In particular, this means that non-breaking spaces (for instance
+  ["\u{A0}"]) are left unchanged.


I think it would be better to talk in Format speak:

Spaces " " (U+0020) are replaced by "@ ".

Newlines "\n" (U+000A) are replaced by "@<0 1>".

The rest is kept unchanged and in particular, non-breaking spaces like U+00A0.

dbuenzli · 2024-10-24T18:30:01Z

+  replaced. In particular, this means that non-breaking spaces (for instance
+  ["\u{A0}"]) are left unchanged.
+- [0<=k<=1] newlines followed by [n] spaces are converted to a [@;<0 (k+n)>]
+  break hint (when [k+n>=1]).


Isn't that rule redundant with the space and newline replacing rules ? If not I'm not sure I understand its implications.

Coalescing multiple spaces in a single break hint ensure that the line is split at most once in the beginning of the group of spaces. This ensures that the spaces in

format_text "a b"

is translated to a single line break rather than multiple line breaks between blank lines.

Ah yes I think I was confused by the sentence. It read as if this happened only after newlines, but k can be 0. I think the overall formulation is more difficult to understand that it could be. Wouldn't something like this work:

First blank lines (lines made only of spaces) are replaced by two forced newlines "@\n@\n". That is they remain blank in the output.

Remaining sequences made of k spaces (U+0020) and newlines (U+000A) are replaced by @;<0 k>. That is they are either k spaces in the output or a single break.

Breaks can be avoided using-no break space characters (U+00A0).

Lines can be forced by using "@\n"

Blank lines by themselves should map to a single @\n, it is the preceding line that must be a \@n too:

Blank lines (lines made only of spaces) are replaced by \@n.

Sequences of spaces and a newline preceding a blank line are replaced by \@n: Blank lines remain blank lines in the output.

Remaining sequences made of k spaces and newlines are replaced by @;<0 k>.

I have added the new description (with a tweak that the space width is at least 1 on break hints) and adapted the implementation to match the behavior and the specification.

(Does @dbuenzli's thumb up mean that he approves the specification you converged towards? If so, please feel free to merge.)

gasche

I can't find a trace of me having approved this PR. I approve on the basis of my own review and also @dbuenzli's.

nojb

LGTM

nojb · 2025-02-08T08:07:20Z

 *)

+val format_text: ('a,'b,'c,'d,'e,'f) format6 -> ('a,'b,'c,'d,'e,'f) format6
+(** [text fmt] replaces spaces and newlines in the format string literal [fmt]


Suggested change

(** [text fmt] replaces spaces and newlines in the format string literal [fmt]

(** [format_text fmt] replaces spaces and newlines in the format string literal [fmt]

nojb · 2025-02-08T08:10:38Z

+          match newlines, spaces with
+          | (0|1), spaces ->
+              let break = Break("", max spaces 1, 0) in
+              String_literal(before, cons ~repeat:1 break len s pos fmt)
+          | bl, _ ->
+              String_literal(
+                before,
+                cons ~repeat:bl Force_newline len s pos fmt
+              )


Suggested change

match newlines, spaces with

| (0|1), spaces ->

let break = Break("", max spaces 1, 0) in

String_literal(before, cons ~repeat:1 break len s pos fmt)

| bl, _ ->

String_literal(

before,

cons ~repeat:bl Force_newline len s pos fmt

)

let repeat, break =

match newlines, spaces with

| (0|1), spaces -> 1, Break("", max spaces 1, 0)

| bl, _ -> bl, Force_newline

in

String_literal(before, cons ~repeat break len s pos fmt)

this functions replaces spaces and newlines ([' '] and ['\n']) by Format formatting hints in format string literals. This make it easier to mix free-flowing text with [Printf] specifiers, boxes and tags.

and simplify the implementation along the way

Octachron added stdlib feature-wish labels Oct 23, 2024

gasche reviewed Oct 23, 2024

View reviewed changes

dbuenzli reviewed Oct 24, 2024

View reviewed changes

Octachron mentioned this pull request Feb 4, 2025

Format: add an out_width function to output device #13570

Merged

gasche approved these changes Feb 4, 2025

View reviewed changes

nojb approved these changes Feb 8, 2025

View reviewed changes

Octachron added 7 commits February 20, 2025 13:47

stdlib: add a format_text function for text-like format strings

828b92d

this functions replaces spaces and newlines ([' '] and ['\n']) by Format formatting hints in format string literals. This make it easier to mix free-flowing text with [Printf] specifiers, boxes and tags.

review: paragraph, tmc implementation and documentation

41a836a

review: remove blank line coalescing, update description

8d639ef

and simplify the implementation along the way

add Changes entry

06a78f6

review: typo in documentation

55a825d

review: simpler format_text breaks computation

6b0b8c3

update Changes

21e840f

Octachron force-pushed the format_text_function branch from d4b6cac to 21e840f Compare February 20, 2025 12:52

Octachron merged commit b019ca2 into ocaml:trunk Feb 20, 2025

	(** [text fmt] replaces spaces and newlines in the format string literal [fmt]
	(** [format_text fmt] replaces spaces and newlines in the format string literal [fmt]

Conversation

Octachron commented Oct 23, 2024

Uh oh!

dbuenzli commented Oct 23, 2024

Uh oh!

gasche commented Oct 23, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Octachron commented Oct 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dbuenzli commented Oct 23, 2024

Uh oh!

Octachron commented Oct 23, 2024

Uh oh!

gasche commented Oct 24, 2024

Uh oh!

dbuenzli left a comment

Choose a reason for hiding this comment

Uh oh!

dbuenzli Oct 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gasche left a comment

Choose a reason for hiding this comment

Uh oh!

nojb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nojb Feb 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Octachron commented Oct 23, 2024 •

edited

Loading

dbuenzli Oct 24, 2024 •

edited

Loading

nojb Feb 8, 2025 •

edited

Loading