Convert Javadoc comments to the /// Markdown style

[yashmayya]

Converts every Javadoc comment in the repo to the JEP 467 Markdown doc comment style (///), and replaces the raw HTML and {@link}/{@code} inline tags inside them with native Markdown.

Why

/// is already the dominant style for new code here (~2,500 files used it before this change), so the codebase had two competing Javadoc styles side by side. This finishes the migration so there is exactly one.

Beyond consistency, a lot of the old /** */ Javadoc was quietly rendering wrong, and converting it surfaced and fixed that:

• Markdown-style lists that never rendered as lists. Plenty of comments were written as - item bullets, which HTML Javadoc flattened into a single run-on paragraph. They now render as actual lists (~250 comments).
• Backticks that rendered as literal backticks. Authors wrote `someMethod()` expecting code formatting and got the backtick characters instead. These are now real code spans (~60 comments).
• < and & in prose that Javadoc flagged as errors. Text like watermarkMs <= 0 or start & end time rendered as a conspicuous invalid input: '<' marker in the generated docs. Those are gone (~67 comments).
• ASCII diagrams mangled by whitespace collapsing. Several diagrams and SQL examples were unreadable because HTML collapsed their alignment; they are now fenced code blocks.
• Hand-rolled <table> markup is now GFM tables, which additionally get proper <thead>/<tbody>.

The /// form is also simply easier to read and edit — no leading-asterisk column to maintain, and no HTML tags interleaved with prose.

What changed

Comment text only. The single non-comment change is the removal of 143 import statements (details below).

Before	After
/** ... */	///
{@link R}, {@link R#m()}	[R], [R#m()]
{@link R label}	[`label`][R]
{@code X}, <code>X</code>	`X`
<b>/<strong>, <i>/<em>	**x**, _x_
<a href="U">T</a>	[T](U)
<ul>/<ol> + <li>	- / 1. lists
<pre>{@code ...}</pre>	``` fenced blocks
<table>	GFM tables
<p>	blank line

The ASF license header at the top of every file is deliberately left as /** */.

A Checkstyle rule is added in config/checkstyle.xml so the codebase does not drift back:

<module name="Regexp">
  <property name="format"
            value="(?m)^[ \t]*/\*\*(?!\*)(?![\s\S]{0,60}?Licensed to the Apache Software Foundation)"/>
  <property name="illegalPattern" value="true"/>
  <property name="message"
            value="Use /// markdown doc comments (JEP 467) instead of /** */ Javadoc."/>
</module>

It reports zero violations across the converted tree, and fails the build with that message the
moment a new /** */ doc comment is introduced. It deliberately does not match the ASF license
header (matched by its text rather than its position, since a few files place it below the
imports), /*** ... ***/ decorative separators, or a /** appearing mid-line inside a string
literal. Generated sources are already covered by the existing entries in
config/suppressions.xml, so they need no special handling.

Verification

Every mapping above was confirmed by rendering it with the JDK 25 javadoc tool and diffing the generated HTML, rather than assumed. {@link R} → [R] and {@code X} → `X` produce byte-identical output; <b>→<strong> and <i>→<em> are the only tag substitutions, and are visually identical.

To check the change end to end, all 15,025 doc comments were rendered twice — once from the original /** */ source and once from the converted Markdown — and the resulting HTML compared. 692 comments (4.6%) render differently, and they break down as:

• ~383 are the fixes listed above (lists, code spans, invalid input markers).
• ~215 are references that cannot resolve in the isolated harness — identical in a real build.
• 24 heading-level shifts. Javadoc remaps Markdown headings into the page outline. ## reproduces the old <h3> exactly in class-level docs; in member docs the heading renders a level or two smaller.
• 19 <br> tags that became paragraph breaks. Markdown has no whitespace-safe hard line break (the CommonMark \ form renders literally in Javadoc), so this one is unavoidable.
• Zero emphasis, link, or list regressions.

Also verified:

• Full reactor mvn test-compile passes — 84 modules, 0 errors — along with spotless:check and license:check.
• No line exceeds the 120-character Checkstyle limit.
• The new Checkstyle rule was verified in both directions: zero violations on the converted tree,
  and a deliberately reintroduced /** */ comment fails the build as expected.
• All 5,830 license headers are intact and untouched.
• Diffing the comment-stripped source against master confirms no code changed anywhere except the import removals below.

Import removals

Checkstyle's UnusedImports only parses /** */ comments, so an import referenced solely from a {@link} inside a /// comment is reported as unused. For those 143 cases the reference was rewritten to its fully-qualified form and the import dropped. This does not affect rendered output — Javadoc renders a reference as its simple name whether or not it is written fully qualified. Worth flagging as a follow-up: if a future Checkstyle release understands Markdown doc comments, these can go back to simple names plus imports.

Two places where raw HTML remains

Both are cases Markdown cannot express:

• 5 links whose URL alone exceeds the line limit. A Markdown link destination cannot contain a newline; the existing code splits these URLs inside <a href="...">, which is the only form that fits.
• 2 wide tables, because a GFM table row cannot be wrapped.

Other notes for reviewers

• 19 /*** banner ***/ separators are left as block comments. Javadoc technically treats them as doc comments, but they are decorative section dividers, not documentation.
• 17 Thrift-generated files under pinot-common/src/main/java/org/apache/pinot/common/request/ and the Thrift test fixtures are excluded — they are regenerated from .thrift sources and are already listed in config/suppressions.xml.
• Characters that were inert in HTML Javadoc but are meaningful in Markdown (*, _, [, ] in text like __name__ or [column_name]) are escaped so they keep rendering literally.
• {@inheritDoc}, {@value}, and {@literal} are left alone, as they have no Markdown equivalent.

One behavioural change worth calling out: JavadocStyle and MissingOverride in config/checkstyle.xml only inspect /** */ comments, so this migration retires those two checks in practice. Both are anchored on the block-comment form and would need updating (or a Checkstyle upgrade) to apply to ///. JavadocStyle mostly guarded against malformed HTML, which matters far less now that the comments are Markdown; MissingOverride is the more useful of the two and is worth a follow-up.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 65.59%. Comparing base (c500e4f) to head (58610fa).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #19126      +/-   ##
============================================
- Coverage     65.60%   65.59%   -0.02%     
  Complexity     1423     1423              
============================================
  Files          3450     3439      -11     
  Lines        218378   218147     -231     
  Branches      34670    34650      -20     
============================================
- Hits         143275   143084     -191     
+ Misses        63538    63506      -32     
+ Partials      11565    11557       -8     

Flag	Coverage Δ
custom-integration1	100.00% <ø> (ø)
integration	100.00% <ø> (ø)
integration1	100.00% <ø> (ø)
integration2	0.00% <ø> (ø)
java-25	65.59% <ø> (-0.02%)	⬇️
temurin	65.59% <ø> (-0.02%)	⬇️
unittests	65.58% <ø> (-0.02%)	⬇️
unittests1	56.96% <ø> (-0.04%)	⬇️
unittests2	37.95% <ø> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[xiangfu0]

can we also add a checkstyle rule for this ?

[yashmayya]

can we also add a checkstyle rule for this ?

A naive one would flag all the ASF license headers, let me check if its possible to add a more sophisticated one.

Edit: added a regex based checkstyle rule - verified that there's no false positives right now, and that it does actually catch new additions of /** style Javadocs.