Added preprocess [default to (page,path) -> page] option

[sylvaticus]

The preprocess function is called after having read the String from
file and before it is parsed as a Markdown page.

[sylvaticus]

ugh, from what I have understood the error doesn't seem to be related to this pull request...

[mortenpi]

I appreciate the PR, but I am skeptical whether we want to add another option to makedocs for this. This can easily be achieved by writing a small loop in the make.jl before you run makedocs. But I am happy to leave this open for a while to get other opinions (@fredrikekre?).

PS: if your concern is modifying source files (which I agree would be bad practice), you can still have your source files in src/, but then have the preprocess loop write the processed files into e.g. src-processed/ and then set the source keyword to makedocs correspondingly.

[sylvaticus]

Hello, I find using a callback much neater. The motivations are exactly those for the equivalent functionality in Literate.jl, just there it applies only to the *.jl -> *.md transformation, while here it would apply to any page whatever its origin is.
Yes I can cp to a temporary directory and preprocess/make documentation there but then the links "Edit on GitHub" would not work, so I need to cp to a temporary directory, work on the original directory and then cp back temporary directory -> original source directory, not something I like much... :-(
As I said, I just feel using a callback is the neatest approach, but again, that's just an opinion...

[mortenpi]

Hmm. I concede the point about the edits links --- it would be nice if those would just work, without having mess around with the EditURL.

@fredrikekre: From the perspective of Literate UX, do you think it would be a good idea to have some way to hook Literate more directly into Documenter? Or do you think keeping them fully decoupled and using intermediate files like we do now is better?

But I am still a little hesitant to introduce this particular API. I do think though that some sort of user-provided parsing of files would be a good thing, but I think it should be generating the AST directly so that it would be more flexible (e.g. I have an ancient experimental branch that allows a different Markdown parser to be used: #1074 (comment)). This type of pre-processing would then just be a special case. But such a feature is much more work.

[sylvaticus]

Just a small comment: If you allow hack at the level of the AST then users would need much more knowledge on the internals of Documenter/Markdown parser compared than just working on a string..

[mortenpi]

That is true, but the user can fall back to using the standard parser and not actually work with the AST. Basically, rather than returning a string s from the callback, you would return e.g. Markdown.parse(s).