Issue 9731: implement $(DDOC_PARAGRAPH)

[quickfur]

Fixes: https://issues.dlang.org/show_bug.cgi?id=9731

Basically, this will wrap text within ddoc comments into proper paragraphs (via a new macro DDOC_PARAGRAPH) so that ddoc output can produce correctly-nested paragraph tags in strict output formats like XHTML or DOCBOOK.

For backward compatibility, DDOC_PARAGRAPH defaults to just $0, and DDOC_BLANKLINE retains its old behaviour. The expected usage pattern, however, is to define DDOC_BLANKLINE and BR to be blank, and DDOC_PARAGRAPH to be <p>$0</p> (or equivalent).

The testcase uses contrived tags for DDOC_PARAGRAPH and BR in order to verify that the generated paragraph tags came from DDOC_PARAGRAPH (not something else that expands to <p>), and that the $(BR) macros are generated in backward-compatible locations.

[andralex]

Ah, this is awesome! Thanks!

I see a paragraph break is consider "\n\n". I think we should make any sequence of whitespace (spaces, tabs, cr, lf) that contains two or more \n a paragraph break. Thoughts?

[andralex]

Thinking about it a bit more I guess it's fine if we leave other whitespace be. I'd say any run of \n containing at least two should be considered one paragraph break.

[quickfur]

Actually, that's what this PR does right now: any runs of \n more than 2 is considered a single paragraph break (see the part of the test case that has multiple line breaks before a new paragraph). I guess this is somewhat obscured by the backward-compatibility with DDOC_BLANKLINE, which still gets inserted multiple times in this case. But the new DDOC_PARAGRAPH only gets a single break.

IOW:

/**
 * First line
 *
 *
 *
 *
 * Second line
 */

produces multiple instances of DDOC_BLANKLINE between the two lines, as backward-compatible behaviour, but only two instances of DDOC_PARAGRAPH for the two lines no matter how many additional blank lines come in between.

The idea is that if DDOC_PARAGRAPH is not overridden, the output is exactly the same as before, but if you override DDOC_PARAGRAPH, then you would also redefine DDOC_BLANKLINE to be empty, so the multiple breaks don't show up in the output.

[andralex]

@quickfur waiting for your comeback!

[adamdruppe]

What, exactly, breaks in the pdf build? This code in property.dd looks wrong to me:

    $(OL
    $(LI If $(D T) is a nested struct, the context pointer in $(D T.init)
    is $(D null).)

----------------
void main()
{
    int a;
    struct S
    {
        void foo() { a = 1; }  // access a variable in enclosing scope
    }
    S s1;           // OK. S() correctly initialize its frame pointer.
    S s2 = S();     // OK. same as s1
    S s3 = S.init;  // Bad. the frame pointer in s3 is null
    s3.foo();       // Access violation
}
----------------

The code example isn't actually in the <li>... it is just next to it. You aren't supposed to do that in html, a <ul> or <ol> tag can only contain <li> and script tags. You also aren't supposed to put a list inside a paragraph... You can put <p> tags inside li's though.

Now, going for generating purely valid html might not be something ddoc can handle with its simple automation, but we should try to do it and part of that would be moving the code example to reside inside the <li> block, then seeing how this PR does again with the build.

BTW we might also set it so the tag is not written out for empty paragraphs. I'm just looking at the code here but I think it currently might output empty tags.