Add documentation tests

[tiehuis]

Summarised from here #452.

• Move docs from https://github.com/zig-lang/www.ziglang.org to this repo
• Make the docs a template that gets processed by a tool written in zig that looks for special "example" code blocks, also with metadata saying whether they should have a compile error, or succeed with tests, or whatever.
• While we're at it, make the table of contents auto generated.
• Run the doc generating tool along with the tests, and have it crash if example code has an unintended effect.
• Do it for the homepage too

[tiehuis]

Instead of performing rudimentary text processing just yet, one thing I was thinking of was to run pandoc on the output and get all code blocks. We can tag code blocks with an attribute like test-should-compile test-should-fail and for each of the blocks run the test command and check the result of the compiler.

We can match the compilation output against test-output blocks that should come after any with the above attribute.

This is a pretty hacky approach and I was messing around with this using pandoc and jq and it seems doable pretty quickly. Alternatively, use something like https://github.com/hoedown/hoedown and build a zig program around this. A bit more work here though and I need to check how/if we can attach attributes to code blocks if following this.

[andrewrk]

I think if we didn't add any new dependencies I'd be up for this going into 0.1.0. If we want to make the build depend on more stuff let's push it to 0.2.0.

[tiehuis]

I think we should aim for doing this without dependencies, but still for the 0.2.0 since it isn't a hugely visible change from the user perspective.

[andrewrk]

Still for the 0.1.0 you mean? I'm ok with that.

[tiehuis]

I meant the 0.2.0, but I'll see where I get with this markdown parser and get back to you.

[andrewrk]

Alright I'm ok with either, you can make the call.

[andrewrk]

@tiehuis would you be terribly offended if I ignored the markdown pull request for now and made a tool in zig that did the above bullet points? I think it would be nice to only depend on zig once we have the compiler built.

[tiehuis]

Not at all, go ahead.

I've started a markdown parser here but it'll be a while before it gets to the state of parsing the entirety of that file anyway.

[andrewrk]

I have this working and integrated into the build system. It generates the table of contents and the "see also" sections, but I haven't made it do code snippets yet.

I'm thinking something like this:

{#code|id_name#}

And then we'd have a table of id_names mapped to source code snippets with metadata.

Or maybe even allow embedding it:

{#code_begin|test#}
test "aoeu" {

}
{#code_end#}

{#code_begin|exe|hello#}
pub fn main() -> %void {}
{#code_end#}

{#code_begin|error#}
test "initialization" {
    var x: i32;

    x = 1;
}
{#code_end#}

[tiehuis]

Great! Embedding would be great. I see the snippets anyway as first and foremost documentation, the testing is just to ensure they are always up to date. We would have a lot anyway and it'd save context switching.