Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate Rust code from the tutorial text #53

Merged
merged 81 commits into from Oct 1, 2019
Merged

Generate Rust code from the tutorial text #53

merged 81 commits into from Oct 1, 2019

Conversation

@tomassedovic
Copy link
Owner

tomassedovic commented May 9, 2019

This updates the Asciidoctor documents to allow generating the final
source files at the end of each chapter directly from the tutorial
text.

The inline source blocks in the docs now contain tags delimiting
sections of the code we want to pull into the final files.

We're adding separate documents that can include these tagged regions
and output the final *.rs files identical to the hardcoded ones in
the src/bin directories.

There are several benefits of generating these Rust files out of the
document itself:

  1. The docs & corresponding sources are always in sync (the doc being
    the single source of truth)
  2. The Rust sources are generating pretty in a similar way as a person
    going over the tutorial would
  3. There is reduced code duplication -- if files 3 to 13 contain the
    same function and we want to change it, there's only one place to do
    that (right now one would have to change it in all 10 Rust files plus the
    tutorial doc)

Downsides:

  1. The files are no longer checked out in the repo so you can't clone
    it and run the examples. We could still generate & check them after the fact, but
    that might confuse potential contributors (they could try & fix those
    files without realising they were autogenerated)
  2. You can't easily fix stuff with tools such as cargo fmt
    automatically any more -- they don't work on adoc which is the single
    source of truth. We could generate RS and run it in the CI (we should
    verify they compile anyway) and fail if cargo fmt would make any
    changes.

Unfortunately, the comments indicating the source tags (regions) are
visible in the HTML generated from adoc. This is clearly undesirable
and I could not figure out how to hide those directly.

The current workaround is: add [role=hide] to the "source-of-truth"
code block (no need to annotate it with [source,rust] any more) --
that will apply the hide CSS class to the block, making it invisible
thanks to the included stylesheet.

And then immediately follow it up with another (visible) Rust source block
include::file.adoc[tag=...] which adds the whole thing and formats it
as Rust. The include process does strip all tags.

This is not great -- it adds a bunch of boiler plate, but it seems to
work fine. I'll try to see if there either is a way to hide the tags
in asciidoctor or if that's something that could be added. But in the
meantime, I think this is a decent workaround (the benefits are imho
awesome).

tomassedovic added 30 commits May 7, 2019
This updates the Asciidoctor documents to allow generating the final
source files at the end of each chapter directly from the tutorial
text.

The inline source blocks in the docs now contain tags delimiting
sections of the code we want to pull into the final files.

We're adding separate documents that can include these tagged regions
and output the final `*.rs` files identical to the hardcoded ones in
the `src/bin` directories.

There are several benefits of generating these Rust files out of the
document itself:

1. The docs & corresponding sources are always in sync (the doc being
the single source of truth)
2. The Rust sources are generating pretty in a similar way as a person
going over the tutorial would
3. There is reduced code duplication -- if files 3 to 13 contain the
same function and we want to change it, there's only one place to do
that (right now one would have to change it in all 10 Rust files plus the
tutorial doc)

Downsides:

1. The files are no longer checked out in the repo so you can't clone
it and run the examples. We could still generate & check them after the fact, but
that might confuse potential contributors (they could try & fix those
files without realising they were autogenerated)
2. You can't easily fix stuff with tools such as `cargo fmt`
automatically any more -- they don't work on adoc which is the single
source of truth. We could generate RS and run it in the CI (we should
verify they compile anyway) and fail if `cargo fmt` would make any
changes.

Unfortunately, the comments indicating the source tags (regions) are
visible in the HTML generated from adoc. This is clearly undesirable
and I could not figure out how to hide those directly.

The current workaround is: add `[role=hide]` to the "source-of-truth"
code block (no need to annotate it with `[source,rust]` any more) --
that will apply the `hide` CSS class to the block, making it invisible
thanks to the included stylesheet.

And then immediately follow it up with another (visible) Rust source block
`include::file.adoc[tag=...]` which adds the whole thing and formats it
as Rust. The `include` process does strip all tags.

This is not great -- it adds a bunch of boiler plate, but it seems to
work fine. I'll try to see if there either is a way to hide the tags
in asciidoctor or if that's something that could be added. But in the
meantime, I think this is a decent workaround (the benefits are imho
awesome).
The general idea is: can we keep the canonical adoc sources in the
`/doc/` directory and then move both the .rs templates as well as the
final adoc bits (e.g. the single include that filters out tags)
elsewhere?

So the user would always see the canonical sources first -- those are
the files most likely to be edited -- and then dig into the templates
only if need be.
This updates the `Makefile` to only build code that's in the output
templates file. It includes a file from a directory above (the
"source" adoc) and produces an identical output HTML page with the
tags stripped out.

I wish we didn't have to basically create these single-line
include-only adoc files, but it matches the source code ones and it
gives us everything we need.
This is the correct name for this tutorial page.
Now all the tutorial text is available again. The 3-13 source code
entries are still missing, though.
This includes some necessary changes to the previous part as well as
adding code snippets that were missing from the tutorial text for some
reason (clearly showing why we're generating the Rust code from the
adoc file).
I thought the tags need to have wrapped in the comments for the source
code, but that's not true.

Instead of:

    // tag::some_code_snippet[]

We can use just:

    tag::some_code_snippet[]

So this removes all those extra prefixes from the source adoc files.
There are two things I couldn't figure out how to do in Asciidoctor
directly: removing the callouts and adding a newline character at the
end of the last line.

So this adds shell commands to do just that in the `docs` make target.
With the merge, we got to a bunch of new conflicts between the manual
and generated Rust sources.

Now they diff cleanly again.
This target will compare the Rust files generated by Asciidoctor
against the hand-written ones in `src/bin`. The goal is to have them
both identical (both by generating the expected code as well as fixing
issues with the hand-written code) before deleting the hand-written
ones.
Some documents, such as the fifth part use the same name for the
tutorial and source listing.

And even though the output will be either html or rs, both "include
template docs" have the same `part-5-combat.adoc` name.

So this splits the adoc includes and rust templates into separate
folders and updates the Makefile to read both.
It's incomplete
The `docker-asciidoctor` container has everything we need:

https://hub.docker.com/r/asciidoctor/docker-asciidoctor/
tomassedovic added 27 commits Sep 15, 2019
We used to use `[Object]` which means one more change to maintain and
explain later on.
Not doing so shows warnings in recent Rust.

Partially resolves issue #297.
It lets us write this:

    use serde::{Deserialize, Serialize};

Instead of:

    #[macro_use]
    extern crate serde_derive;
Mostly `game.log` -> `game.messages` and setting the order of function
calls to: tcod, game, objects.
Keep the right order of arguments (tcod, objects), remove the
unnecessary `mut`.
I don't really like it -- there's a lot of "add attack stuff, then
compile, then handwave the rest".

It could probably benefit from some restructuring tbh. Like introduce
all the bases (e.g. `base_attack`, `base_defense`, `base_max_hp`) in
one go and then build on top of that.
Fixed tcod version in Cargo.toml, updated the wrong callout syntax in
adoc files, etc.
All the the Rust files are now autogenerated. As such, we should add
that information to their header.
It builds the Rust files from the docs and copies them to `src/bin/`.
The contents of that folder is autogenerated and this is how we do it.
I just ran `make update-cargo-bin`.
Described how to generate the tutorial documents and Rust files as
well as how to contribute and the maintenance status of the project.
@tomassedovic tomassedovic changed the title WIP POC Generate Rust code from the tutorial text Generate Rust code from the tutorial text Oct 1, 2019
@tomassedovic tomassedovic merged commit d4bf748 into master Oct 1, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.