Blank lines in comments

[matklad]

{% This is a comment, spanning
multiple lines 

It also cintains a blank line
%}

doc
  para
  para
    str s="multiple lines"
  para
    str s="It also cintains a blank line"
    softbreak
    str s="%}"

It feels like blank lines in comments is a rather reasonable thing to want.

Note that we don't support blanks in general attribute syntax (ie, we allow newlines, but not blank lines) but that I think is reasonable.

[jgm]

Agreed that blank lines in comments are reasonable to want.

Actually I think the attribute parser will accept blanks; the problem is that by the time the (inline) attribute parser gets this, the content has already been split into paragraphs. (Block-level recognition is prior to inline-level.) So this one is a bit tricky.

[matklad]

It's curious that this literal example can be seen as a block comment. That is, block parser can treat {% and %} like :::.

And it seems reasonable to say that only block comments can have blanks in them

[jgm]

Block attributes must fit on one line. So you can't use % in a block attribute if your comment has a blank, nor can you use % in an inline attribute, because the blank line will start a new paragraph.
You could do something like

{% This is a comment, spanning }
{% multiple lines }
{% }
{% It also contains a blank line }

But that's not convenient or beautiful.

It would be possible to introduce a block comment fenced syntax starting with {% and ending with %} but I think there is some danger of confusion because of the overlap with attribute syntax.

[waldyrious]

Not sure if I'm missing something obvious, but the solution that comes to my mind here would be to use %%% as syntax for block comments:

%%%
This is a comment, spanning
multiple lines 

It also contains a blank line
%%%

[jgm]

I don't much like this style of fenced comments, since the open and close delimiters aren't marked as such; it makes it difficult to do things like comment out sections that already have comments.

I'd prefer a line-mark syntax, which makes it evident to the eye what is commented out.

[uvtc]

Since djot already supports $`a+b=c` for inline LaTeX math, and $$`d=ef` for display math, maybe

%`this for an inline comment`

and

%%`This for
a multi-line comment.

Contains a blank line.`

(fixed typo)

[jgm]

Also worth considering

%`this is inline`

%```
This is a block

comment which can contain a blank line.
```

Note that inline verbatim can't contain blank lines. If we did that we'd probably want to make the math syntax parallel, see #160 (comment)

In general, though, I prefer a "line comment" style, like LaTeX %. This makes it much more evident to the eye which parts are commented out. For example, something like

%% These lines are
%% commented out
And this isn't%% except for the last part

This is also easy for text editors to highlight, and highlighting comments is one of the basic things you want to be able to do even if you're not highlighting much else.

[tbdalgaard]

Like the idea of {% and %} regardless of what the comment contains.

I think it is important that Djot can be used by non programmers who just want some easy to follow syntax rules, and that is exactly why I like Djot so much as I do. I am trying to change all my daily work from Markdown to Djot, because of the syntax and given how easy it is to adapt.

[bpj]

So the syntax is simply this?

LINE %% COMMENT

Like in TeX but (at least) two % chars? I like it. KISS principle!

[jgm]

Well, that's one idea anyway!

[mcookly]

I'm not sure how much of a factor syntax highlighting will be. However, using "line comment" style lets the reader know quickly whether or not a line is a comment, even when no syntax highlighting is present. Having the option to easily read a djot document without syntax highlighting is definitely handy.

[Omikhleia]

Just my 2 cents after finding this issue, but the current situation with this asymmetry is indeed not really friendly. There might be a good rationale for it, but it feels weird:

Lorem{% OK comment %}

Lorem{% OK
comment
on multiple
lines %}

{% OK comment %}

{% NOK not a comment
currently %}

{% OK block level comment %}
> block

{% NOK not
a comment
currently %}
> not even a block!

[jgm]

It was probably a mistake to rely on the attribute comment syntax for comments; as noted above, I think a proper line comment syntax like LaTeX's % would be better. Indeed, I'm not even sure a comment syntax in attributes is needed.

[mcookly]

I prefer LaTeX's comment syntax as well. That being said, if someone writing prose in djot (which probably means paragraph/line) wants to include comments in the middle of their paragraph, they can't use LaTeX comment style, right?

[jgm]

That being said, if someone writing prose in djot (which probably means paragraph/line) wants to include comments in the middle of their paragraph, they can't use LaTeX comment style, right?

Sure they % comment here
can.

[bpj]

As I found for Pandoc Markdown you can already (ab)use raw block/inline syntax for persistent comments as long as the “format” doesn’t coincide with any format actually in use:

```=comment
This is
a multi-line
comment.
```

This is `a comment within`{=comment} a paragraph.

Perhaps since the “format” in raw block/inline syntax may be/begin with an underscore in both djot and Pandoc markdown: assuming no actual format name starts with an underscore names beginning with an underscore may by convention be used for application-specific raw-text based extensions which will degrade gracefully by being ignored, and by further convention the lone underscore as format could be for comments/always ignored stuff. An application or filter could do anything with these, for example parse the text with its own parser and construct elements to insert in its place.

```=_
This is
a multi-line
comment.
```

This is `a comment within`{=_} a paragraph.

```=_csv_file
my-table.csv
```

```=_csv
foo,bar,baz
tic,tac,toc
```

Just my two small coins!

[mcookly]

if someone writing prose in djot (which probably means paragraph/line) wants to include comments in the middle of their paragraph, they can't use LaTeX comment style

The term is soft-wrapping. I forgot the word 🤦‍♂️

LaTeX comment style does not allow comments in the middle of a soft-wrapped paragraph, i.e. a single line. I think it would be good to have another comment option for those who use prefer having one line be a whole paragraph of text.

[rauschma]

It was probably a mistake to rely on the attribute comment syntax for comments; as noted above, I think a proper line comment syntax like LaTeX's % would be better. Indeed, I'm not even sure a comment syntax in attributes is needed.

I feel like this may already be implied by some of the comments but wanted to state it explicitly: If there were no comment syntax in attributes then the syntax for comments would not be tied to the syntax of attributes anymore and djot could support the following (reminiscent of how CommonMark handles comments).

Some text {% inline comment (no empty line allowed) %}

{%
block comment

with empty line
%}

Line comment syntax would be complementary (similar to /* */ and // in C++ etc.):

%% This is a line comment

[jgm]

Yes, it is certainly tempting to remove the comment syntax from attributes and just assign {%..%} as comment syntax. (An unclosed {% would just capture everything up to the end of the enclosing block I suppose, or perhaps the end of input.)

[bpj]

Yes, it is certainly tempting to remove the comment syntax from attributes and just assign {%..%} as comment syntax. (An unclosed {% would just capture everything up to the end of the enclosing block I suppose, or perhaps the end of input.)

Yes please make {%...%} the comment syntax, perhaps with %%%...%%% as block comment syntax so as to avoid that paragraphs containing only a comment become an issue. I don't know if I mentioned it before, but the primitive Perl script I used before I found Markdown used {%...%} for comments and it became an issue there as did comments with blank lines in them (something I forgot about constantly! :-) I'm divided on whether unclosed comments should continue to the end of block or the end of input. The former seems less "damaging", but the latter is more noticeable, which may actually be better.

[jgm]

Actually, now that I think about it, it's not really straightforward to make {% ... %} comment. The reason is that we parse blocks before looking at inline content. So, if you have

test {% and this

is commented %}

the parser will first identify two paragraphs (just based on the start of the lines) and the comment delimiters would only be seen at the inline parsing phase. But in that phase we're just looking at one paragraph, so we don't have the matched pair.

Line comments are unproblematic, though.

Q: Couldn't we just change the parser so that it scans each text line for {% before adding it to the block; if {% is found, it enters a special comment mode and only exits it when %} is encountered?

A: It's not that simple. After all, the {% may occur in a literal context (e.g. inline verbatim). So without actually parsing the line we couldn't tell. And even then we can't tell on a line by line basis; maybe the beginning of the inline verbatim was on a previous line. I don't see a clean way to do this with the current parsing architecture, which splits up block parsing and inline parsing.

Indeed, even a line comment syntax is tricky to implement, for the same reason. We'd have to handle the comment syntax (say %%) either in block parsing or in the inline parsing phase. If we try to handle it in the block parsing phase, then we face the aforementioned problem about occurrences of %% in literal contexts (verbatim, URLs). If we try to handle it in the inline parsing phase, we run into other problems: for example, this would turn into an empty paragraph:

%% foo
%%
%% bar

and I don't know how we'd handle this, since here the line comments are not in an inline content context:

%% 1. list
%% 2. commented out
3. item

[rauschma]

Couldn’t this be handled like CommonMark handles <!-- -->?

• Inline comment: <!-- appears inside a line. The comment is parsed as an inline. An empty line terminates it. It extends at most until the end of its paragraph.

• Block comment: <!-- starts a line. The comment is parsed as a block – in a manner loosely similarly to code blocks.

[jgm]

I suppose we could have handlers for %% in both block and inline parsing phases; in block contexts only initial %% would be significant.

As for {% and %}, yes, it would be possible to treat them differently in block and inline contexts, as with HTML comments in commonmark. I fear, though, that this would be confusing. People would wonder why

{% foo

bar %}

comments out the whole section while

test {% foo

bar %}

does not.

[bpj]

@jgm that's why I think it is better to have {%...%} for inline comments only and %%%...%%% for fenced block comments. It would anyway be very confusing to use one of them for comments and the other one for something else so you might as well use both for inline and block comments respectively.