Tests for code samples

[verytactical]

Code samples must be executable.

Code samples in changed parts of docs should be re-checked by CI.

[reveloper]

Can we add this to the first release?

[novusnota]

Not all code samples are self-contained: we shall either provide more surrounding context for them somehow (in code comments above/below?) or focus on checking only those examples we can thoroughly check (at least) the syntax of.

The latter is how the checking was done for https://docs.tact-lang.org/cookbook — all examples there contain everything used in order to please the type checker. However, if we apply the same ideas to non-TON languages, we might not be able to execute code directly due to lack of necessary libs for those code snippets. That can be mitigated with Deno-like import specifications or knowing which libraries to use in advance for the whole page, but that still ultimately limits snippets to a single file.

TL;DR: not that easy of a task when one approaches it with careful thought, imo :)

[verytactical]

I wish we could, but realistically this is a huge amount of work. Even if we agree to include merely a single test per snippet (just to formally pass the requirement), running TON-specific tests from markdown sources is not something mintlify supports out of the box, and we'll need some serious development effort to get it working and convenient.

[verytactical]

For example, issue description above mentions "changed parts". For MDX that'd require

• computing diff to merge base
• extracting list of changed files (after we figure out what to do with all the moved/removed files)
• using some MDX API to extract list of imports
• resolving them, and building a dependency graph
• computing transitive closure of changed subgraph
• matching files in the closure to test files, building a test plan
• running it, for example, with jest

While we could assign someone to do this (for example, me), we'd have one less person on hackathon doing what's needed the most: documentation.

[reveloper]

I know that's a costly and unclear issue, just wanted to highlight that this is really a high-priority feature, which may help to keep high quality in the documentation for the long term.

Is it worth the effort instead of creating a one-day living, perfect documentation version.