-
Notifications
You must be signed in to change notification settings - Fork 172
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Please clarify Preprocessor documentation #1262
Comments
In truth, the preprocessor extension should never be used to read lines from the reader (unless they are lines that proceed the start of the AsciiDoc document, such as front matter). Although the prepreprocessor was originally designed to allow this use case, we learned after implementing it that it causes side effects that cannot be easily worked around. The main issue is that when you read lines from the reader in a preprocessor, it blows by the header without processing it. This means it starts processing lines without considering any attributes defined in the document header. Although there's way to do this correctly, it requires a lot of extra code and that code ends up taking on parsing responsibilities. Only someone who is deeply familiar with how AsciiDoc parsed should attempt to do something like this. The preprocessor can be used as an event listener and can safely modify options and attributes on the document. However, it should never read lines from the reader (except for front matter lines, as already mentioned). |
lines is just the raw AsciiDoc source split into lines. It's safe to look at it. And that method is not partially working. It's doing exactly what it is supposed to do, which is to return the source of the current document. The source doesn't know anything about includes and thus isn't going to give you those documents. The |
I'm sorry I was unclear. |
The current Preprocessor API in AsciidoctorJ unfortunately requires to work on the Reader that was passed as an argument. Since there were complaints about breaking changes in a major release, I have no idea atm when I will be able to resolve this and create a new major release. |
...and again, because it's not supposed to. The From my perspective, no change is needed to AsciidoctorJ. We are simply discussing how Asciidoctor works. |
I did notice that the Reader in AsciidoctorJ does not have methods for getSource() and getSourceLines(), which is truly the raw source and not just the remaining lines. Those should probably be added, though that's a separate issue. |
...and a preprocessor is not called for included files because they are not themselves documents. The only time the preprocessor is called is just before the input document is parsed. It has no relationship to includes. |
Yes, this issue is only about improving the documentation, not about changing the implementation. :-)
Maybe this could also be changed with the changes @robertpanzer mentioned. :-) |
That's what the include processor is for, not the preprocessor. |
The docs at https://docs.asciidoctor.org/asciidoctorj/latest/extensions/preprocessor/ suggest that a preprocessor is working like I would expect it to and that it is meant for reading all lines and transforming them before the syntax is then further processed and parsed.
The example also shows exactly that.
But @mojavelinux in Zulip said quite the opposite and the canonical preprocessor page at https://docs.asciidoctor.org/asciidoctor/latest/extensions/preprocessor/ also objects.
You should not read from the reader in a preprocessor in its current state as it will break things further on and even using
lines()
only patially works as the preprocessor extension is only called on the main document but not on any included documents.Please update the docs to make it clear that what is currently described and also shown in the example is not what you should do.
The text was updated successfully, but these errors were encountered: