Accept additional user-defined classes in fenced code blocks

[poliorcetics]

Fix #78917.

This allow users to write the following:

/// Some code block with `rust,class:test:name` as the language string
///
/// ```rust,class:test:name
/// int main(void) {
///     return 0;
/// }
/// ```
fn main() {}

This will notably produce the following HTML:

<pre class="rust test:name">
int main(void) {
    return 0;
}</pre>

Instead of:

<pre class="rust rust-example-rendered">
<span class="ident">int</span> <span class="ident">main</span>(<span class="ident">void</span>) {
    <span class="kw">return</span> <span class="number">0</span>;
}
</pre>

TODO:

• Implement basic functionality
• Write tests
• Document the feature
• Make the feature nightly only (example PR: Stabilize doc_alias feature #75740)
• Clean up the code a little because my changes are very raw right now
• Clean up the tests, they are starting to get unwieldy (Accept additional user-defined classes in fenced code blocks #79454 (comment))

MERGE:

• wait for [librustdoc] Only split lang string on ,, , and \t #78429

[rust-highfive]

r? @jyn514

(rust-highfive has picked a reviewer for you, use r? to override)

[jyn514]

I'm not sure I understand what this is actually getting us. There's no more rust syntax highlighting, yes - but there's no C source highlighting either, which was kind of the point. And the C highlighting can't use the same mechanism, because rust highlighting uses rustc_lexer directly and I do not want to tie rustdoc into clang, nor have to do something similar for each language. That means highlight.js or something like it, but that seems somewhat complicated ...

Anyway, I think we should have this feature actually do something before we accept the syntax, especially since the new feature is insta-stable currently.

[jyn514]

cc @GuillaumeGomez - do you have opinons on highlight.js?

[Hywan]

I think it's better to not force a specific Javascript library to highlight the code. One may want highlight.js while another one may prefer prism.js. The primary goal of this PR is to allow passing any HTML class to the code block and to disable the Rust highlighting in this specific case.

[GuillaumeGomez]

I'm not sure I understand what this is actually getting us. There's no more rust syntax highlighting, yes - but there's no C source highlighting either, which was kind of the point. And the C highlighting can't use the same mechanism, because rust highlighting uses rustc_lexer directly and I do not want to tie rustdoc into clang, nor have to do something similar for each language. That means highlight.js or something like it, but that seems somewhat complicated ...

That was never the goal. The point here is to allow users to add a class themselves and disable the rustdoc highlighting so that they can then add a JS library which will do it (like highlight.js or prism.js as mentionned by @Hywan).

Anyway, I think we should have this feature actually do something before we accept the syntax, especially since the new feature is insta-stable currently.

We can make it remain unstable by adding a check when parsing the codeblocks' tags for the moment. I think it would be preferrable in fact. Users will still be able to use it on docs.rs if they want to and that will allow to make breaking changes if needed.

@poliorcetics Can you add the check for nightly and only allow this feature then? (And otherwise, maybe print a warning saying it's nightly only for the moment?)

[jyn514]

That was never the goal. The point here is to allow users to add a class themselves and disable the rustdoc highlighting so that they can then add a JS library which will do it.

Ahhh, that makes way more sense. Ok, I'm on board with this then.

@poliorcetics Can you add the check for nightly and only allow this feature then? (And otherwise, maybe print a warning saying it's nightly only for the moment?)

This should go through the normal feature gate mechanism, which gives an error when you try to use nightly features on stable.

Another thing is that I think this will change how language strings are parsed, so it should probably wait for #78429.

[poliorcetics]

This should go through the normal feature gate mechanism, which gives an error when you try to use nightly features on stable.

I don't know how those work for rustdoc, is there some place can I find the information or an example ?

[GuillaumeGomez]

@poliorcetics It's the same as for the rest of the compiler. You can see everything that you need in this PR for example (which is removing the feature gate part since it's stabilizing but you got the idea).

[poliorcetics]

With those two commits there are now tests for this ! Still a lot of things to do though

[camelid]

I saw on Zulip that you were going to call this feature user_classes_in_docs. Might custom_code_classes (maybe add docs somewhere, not sure if it's necessary) be a clearer name?

[poliorcetics]

It can easily be renamed as long as this isn't merged. I used a name because I needed one to proceed but I'm not particularly attached to it

[camelid]

tidy error: Found 1 features without a gate test.
Expected a gate test for the feature 'custom_code_classes_in_docs'.
Hint: create a failing test file named 'feature-gate-custom_code_classes_in_docs.rs'
      in the 'ui' test suite, with its failures due to
      missing usage of `#![feature(custom_code_classes_in_docs)]`.
Hint: If you already have such a test and don't want to rename it,
      you can also add a // gate-test-custom_code_classes_in_docs line to the test file.

[GuillaumeGomez]

@poliorcetics Look at the PR I linked above. Everything that I removed in it is was you need to add to have a full feature.

[poliorcetics]

Barring errors in CI, this should now only need clear up work !

[poliorcetics]

Only the tests to clean up now (#79454 (comment))

[camelid]

No longer blocked! #78429 has been merged. 🎉

@rustbot label: -S-blocked

[poliorcetics]

@rustbot label: +S-waiting-on-review -S-waiting-on-author

[jyn514]

This is still waiting on #79454 (comment) and #79454 (comment) (although I don't care very much about the second one).

[jyn514]

Hmm, what does this do? I would expect class: to give an error (or at least a warning).

[crlf0710]

@poliorcetics Ping from triage! Could you address the comments above? Thanks!

[jyn514]

Closing due to inactivity.