Skip to content

[feature](plugin) replace the always-true connector apiVersion() with a manifest-carried API version gate for all four plugin families - #66211

Merged
morningman merged 5 commits into
apache:branch-catalog-spifrom
morningman:catalog-spi-review-22
Jul 29, 2026
Merged

[feature](plugin) replace the always-true connector apiVersion() with a manifest-carried API version gate for all four plugin families#66211
morningman merged 5 commits into
apache:branch-catalog-spifrom
morningman:catalog-spi-review-22

Conversation

@morningman

@morningman morningman commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: #65185

Problem Summary:

Part of the catalog-SPI migration (#65185). Review round 22, targeting branch-catalog-spi.

One self-contained work line: give the four plugin families — CONNECTOR, FILESYSTEM, AUTHENTICATION,
LINEAGE — an API version check that can actually reject a plugin. The design, the version-change
runbook and the eight-task implementation plan are included under plan-doc/designs/.

1. The check that existed was true by construction

ConnectorProvider.apiVersion() could never reject anything. The SPI interface is listed
parent-first, every plugin zip excludes the SPI jar, and no shipped connector overrides the method —
so the default body executed on a plugin's behalf is the kernel's own code, and the number it
returns is the kernel talking to itself. Raising CURRENT_API_VERSION and the default together
admits every stale plugin; raising only the constant rejects every plugin, including freshly built
ones. Neither is a check. The other three families had none at all. apiVersion(),
CURRENT_API_VERSION and the three comparisons against it are deleted.

2. One number, physically distributed to both sides of the comparison

Each family declares its API version once, as a maven property in its parent pom (fe/fe-connector,
fe/fe-filesystem, fe/fe-authentication, and fe/fe-core for LINEAGE). That single property flows
to:

  • a maven-filtered resource inside the family's SPI module — what the kernel expects;
  • the <manifestEntries> of every jar built from that pom — what a plugin declares.

Same build, same property, so the two can never disagree by accident; across builds they disagree
exactly when they should. There is no second number, so no test has to keep them in sync. All four
families start at 1.0.

Families are derived, not enumerated: the token connector implies the kernel resource
/META-INF/doris/connector-plugin-api-version.properties and the manifest attribute
Doris-Connector-Plugin-Api-Version. A fifth family follows the convention on both sides and needs
no new code in the gate.

3. The gate

ApiVersionGate (new, in fe-extension-loader, family-neutral) compares MAJOR only. Minor and
patch are ignored in both directions, which is sound only because "major" is defined as any change
to the SPI surface, additions included — a compatible minor can never change the set of API elements a
plugin can reach. A jar that declares nothing is refused: fail-closed is what makes the check
meaningful for plugins built before this existed.

DirectoryPluginRuntimeManager#loadAll takes the gate as a mandatory fifth parameter, not an
optional one — an unwired gate would be the same silently-true check this PR removes. It runs after
loadClass but before asSubclass/newInstance, so an incompatible plugin's constructor never
executes and the operator gets a message naming both versions instead of a ClassCastException.
Directory loading is the only production path and classpath ServiceLoader builtins never pass
through it, so they are exempt structurally rather than by a flag.

All four families are wired. LINEAGE loads at start-up, and one refused plugin still cannot stop FE.
AUTHENTICATION loads lazily and reports "no factory for this type" from a different place than the
load, so the rejection reason is carried into that exception at both call sites — otherwise a refused
plugin is indistinguishable from one that was never installed.

4. Four SPI surface baselines

One per family, freezing what that family's plugins implement or call — including fe-extension-spi's
Plugin / PluginFactory / PluginContext in all four, so a change there turns all four red at
once. Signatures are recorded with their return type: a changed return type is a major change by
the same definition, and the older name-and-parameters-only baseline (connector-metadata-methods.txt)
cannot see it. No test can prove someone actually bumped the property — a test sees the current state,
never the delta — so the failure message says, in the same commit as the diff, that this is a major
change. The four baselines cannot carry a license header (every non-blank line is read back as a
signature), so they join connector-metadata-methods.txt in .licenserc.yaml.

5. Two build facts, both found by trying rather than by reasoning

  • maven-build-cache hashes a module from its sources, dependencies and <build><plugins>, NOT from
    <properties>
    — and the filtered resource's source text is the literal placeholder. AUTHENTICATION
    and LINEAGE were going to ship with the property alone, since they have no in-tree plugin zips, which
    would have put their version outside every hashed input. Bumping 1.0 → 2.0 then produced an identical
    module checksum (4c1d34a6e04836c6 both times) and a cached jar still declaring api.version=1.0.
    All four families now carry the <manifestEntries> block, and the checksum tracks the bump
    (a388a287b5a7e172cfb1d83e91b92b98). Worth recording how this was caught: adversarial review
    out-voted it 2:1 as "not a defect" on the reasoning that a major bump always comes with an SPI surface
    change — the one reviewer who actually ran the A/B build was the one who was right.
  • fe-connector-es and fe-connector-trino never excluded fe-filesystem-api from their plugin zips,
    shipping a duplicate that parent-first loading guarantees is never read. Now excluded, so all eight
    connector zips agree.

Plan docs are updated to match: D-009 ("the API version never bumps within this plan") is marked
superseded, and the master plan's §2.3 rule is replaced — the bump criterion is now "any SPI surface
change, additions included", not "signature change or method removal".

Release note

None

Check List (For Author)

  • Test
    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

Local verification, stated at the coverage it actually has:

  • full reactor package with the build cache disabled and test sources compiled
    (-Dmaven.build.cache.enabled=false, no -Dmaven.test.skip): BUILD SUCCESS. test-compile is not
    sufficient here — fe-connector-hms compiles against a shade artifact that binds to package — and
    the cache must be off for the reason in item 5.

  • checkstyle:check on all six touched modules: 0 violations.

  • tests: fe-extension-loader 19 (the gate's decision rule, and the loader's behaviour on real plugin
    jars assembled by the test with and without the attribute); fe-core 56 targeted (each family's
    wiring); authentication handler 20.

  • the produced artifacts were read back rather than assumed: doris-fe-connector-es.jar carries
    Doris-Connector-Plugin-Api-Version: 1.0 alongside the inherited Implementation-* entries, and
    doris-fe-connector-spi.jar carries api.version=1.0 rather than an unfiltered placeholder.

  • no e2e was run (no cluster available here) and this PR adds no regression case. The behaviour that
    would want one — an operator's pre-existing plugin zip being refused — is covered by the unit tests
    that build real jars, not by a deployment test.

  • no BE file is touched by this PR, so BE was not rebuilt.

  • Behavior changed:

    • No.
    • Yes.

A directory-loaded plugin jar that declares no API version, or one whose major differs from this FE's,
is now refused at load time; previously nothing was ever refused. Fail-closed is deliberate — accepting
an undeclared version would leave the check meaningless for exactly the plugins built before it existed
— and rebuilding the plugin against this tree is the fix. In-tree plugins need no change: they inherit
the property from their family's parent pom, so the number appears in exactly one place. Classpath
ServiceLoader builtins are unaffected, since they do not go through directory loading.

  • Does this need documentation?
    • No.
    • Yes.

Not on the website yet, and not in this PR. Third-party plugin authors now have to stamp one manifest
attribute, so this becomes user-facing documentation at the point where external plugins are documented
at all — which this branch has not reached. Until then the runbook lives with the design:
plan-doc/designs/2026-07-29-plugin-api-version-check-design.md covers how to tell major from minor,
the Doris-side steps for each, what a third-party author must write, and how to diagnose a plugin that
failed to load.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

morningman and others added 5 commits July 29, 2026 13:05
…gin families

The connector family already carries an apiVersion() gate, but it can never
reject anything: no ConnectorProvider overrides the default method, and the
SPI interface is excluded from every plugin zip and loaded parent-first, so
the default body executing at runtime is the kernel's own. The number never
leaves the kernel. The other three families have no version check at all.

Record the agreed design: the version travels in the plugin jar MANIFEST,
derived from a single maven property per family that also feeds a filtered
resource in that family's SPI module, so the kernel's expectation and the
plugin's declaration cannot drift within a build. Major must match; minor
and patch are ignored, which is safe because every SPI surface change is a
major change by definition.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…dently

The version model already gave each family its own maven property, but the
spec buried that in a table and only mentioned the shared-artifact coupling
in passing. State it directly: changing fe-connector-api bumps CONNECTOR
alone, and the filesystem, authentication and lineage plugins need no
rebuild.

Add the matrix of which artifact change bumps which families, and record why
the two remaining couplings cannot be designed away: fe-extension-spi is
mandatory parent-first for all four families, and the hive, iceberg and
paimon connectors genuinely import fe-filesystem-api types. A separate
version number for those artifacts would describe the same coupling without
removing it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…n design

The design said what the version model is but not what anyone does when the
SPI actually changes. Add a runbook: how to tell major from minor, the exact
Doris-side steps for each, what in-tree plugins need (nothing - they inherit
the property from the family parent pom, so the number appears in exactly one
place), what a third-party plugin author has to write, and how to diagnose a
plugin that failed to load.

Note the two ways to read the kernel's expected version, and that declaring
too low a version fails safe while declaring too high does not - the gate only
guarantees that writing nothing cannot pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…sion gate

Eight tasks, each ending in an independently testable deliverable: the
ApiVersion value type, the family-neutral gate, the loader wiring, then one
task per plugin family, then the contract-surface baselines.

The pom edits are written against what each file actually contains rather than
conditionally - fe-connector-spi and fe-filesystem-spi have a build section
without resources, both authentication poms have no build section at all, and
fe-core already has a resources block that must be replaced wholesale so the
antlr generated-sources entry survives.

Task 8 includes a mutation step that adds a throwaway default method to
ConnectorProvider and asserts the baseline goes red, because those baselines
are the only guard that an SPI surface change gets a major bump.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…eaves the kernel

`ConnectorProvider.apiVersion()` could never reject anything. The SPI interface is
listed parent-first, every plugin zip excludes the SPI jar, and no shipped connector
overrides the method -- so the `default` body executed on a plugin's behalf is the
kernel's own code, and the number it returns is the kernel talking to itself. Raising
`CURRENT_API_VERSION` and the default together admits every stale plugin; raising only
the constant rejects every plugin including freshly built ones. Neither is a check.
The other three plugin families had none at all.

Replace it with one number that is physically distributed to both sides of the
comparison. Each family declares its API version once, as a maven property in its
parent pom, and that single property flows to:

  - a maven-filtered resource inside the family's SPI module -- what the kernel expects;
  - the `<manifestEntries>` of every jar built from that pom -- what a plugin declares.

Same build, same property, so the two can never disagree by accident; across builds
they disagree exactly when they should. There is no second number, so no test has to
keep them in sync.

`ApiVersionGate` (fe-extension-loader, family-neutral) compares MAJOR only. Minor and
patch are ignored in both directions, which is sound only because "major" is defined as
ANY change to the SPI surface, additions included -- a compatible minor can never change
the set of API elements a plugin can reach. A jar that declares nothing is refused:
fail-closed is what makes the check meaningful for plugins built before this existed.
Directory loading is the only production path, and classpath ServiceLoader builtins
never pass through it, so they are exempt structurally rather than by a flag.

`DirectoryPluginRuntimeManager#loadAll` takes the gate as a mandatory fifth parameter.
Not an optional one: an unwired gate would be the same silently-true check this commit
removes. It runs after `loadClass` but before `asSubclass`/`newInstance`, so an
incompatible plugin's constructor never executes and the operator gets a message naming
both versions instead of a `ClassCastException`.

All four families are wired: CONNECTOR, FILESYSTEM, LINEAGE (start-up load, one bad
plugin still cannot stop FE) and AUTHENTICATION. Authentication loads lazily and reports
"no factory for this type" from a different place than the load, so the rejection reason
is carried into that exception at both call sites -- otherwise a refused plugin is
indistinguishable from one that was never installed.

Four surface baselines freeze what each family's plugins implement or call, including
`fe-extension-spi`'s `Plugin`/`PluginFactory`/`PluginContext` in all four, so a change
there turns all four red at once. Signatures are recorded with their return type: a
changed return type is a major change by the same definition, and the older
name-and-parameters-only baseline cannot see it. No test can prove someone actually
bumped the property -- a test sees the current state, never the delta -- so the failure
message says, in the same commit as the diff, that this is a major change.

Two build facts worth recording, both found by trying rather than reasoning:

  - maven-build-cache hashes a module from its sources, dependencies and
    `<build><plugins>`, NOT from `<properties>`, and the filtered resource's source text
    is the literal placeholder. AUTHENTICATION and LINEAGE were going to ship without
    `<manifestEntries>` because they have no in-tree plugin zips -- which would have put
    their version outside every hashed input. Bumping 1.0 to 2.0 then produced an
    identical checksum and a cached jar still declaring 1.0. All four families now carry
    the block.
  - `fe-connector-es` and `-trino` never excluded `fe-filesystem-api` from their plugin
    zips, shipping a duplicate that parent-first loading guarantees is never read. Now
    excluded, so all eight connector zips agree.

The four recorded baselines cannot carry a license header -- every non-blank line is
read back as a signature -- so they join `connector-metadata-methods.txt` in
`.licenserc.yaml`.

Verified: full reactor `package` with the build cache disabled and test sources
compiled; checkstyle clean on all six touched modules; the gate's decision rule, the
loader's behaviour on real plugin jars, and each family's wiring covered by tests; and
the produced artifacts read back -- `doris-fe-connector-es.jar` carries
`Doris-Connector-Plugin-Api-Version: 1.0` alongside the inherited `Implementation-*`
entries, and `doris-fe-connector-spi.jar` carries `api.version=1.0` rather than an
unfiltered placeholder.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman morningman changed the title Catalog spi review 22 [feature](plugin) replace the always-true connector apiVersion() with a manifest-carried API version gate for all four plugin families Jul 29, 2026
@morningman
morningman merged commit 09f86f0 into apache:branch-catalog-spi Jul 29, 2026
42 of 48 checks passed
morningman added a commit that referenced this pull request Jul 29, 2026
… a manifest-carried API version gate for all four plugin families (#66211)

### What problem does this PR solve?

Issue Number: #65185

Problem Summary:

Part of the catalog-SPI migration (#65185). Review round 22, targeting
`branch-catalog-spi`.

One self-contained work line: give the four plugin families — CONNECTOR,
FILESYSTEM, AUTHENTICATION,
LINEAGE — an API version check that can actually reject a plugin. The
design, the version-change
runbook and the eight-task implementation plan are included under
`plan-doc/designs/`.

**1. The check that existed was true by construction**

`ConnectorProvider.apiVersion()` could never reject anything. The SPI
interface is listed
parent-first, every plugin zip excludes the SPI jar, and no shipped
connector overrides the method —
so the `default` body executed on a plugin's behalf is the kernel's own
code, and the number it
returns is the kernel talking to itself. Raising `CURRENT_API_VERSION`
and the default together
admits every stale plugin; raising only the constant rejects every
plugin, including freshly built
ones. Neither is a check. The other three families had none at all.
`apiVersion()`,
`CURRENT_API_VERSION` and the three comparisons against it are deleted.

**2. One number, physically distributed to both sides of the
comparison**

Each family declares its API version once, as a maven property in its
parent pom (`fe/fe-connector`,
`fe/fe-filesystem`, `fe/fe-authentication`, and `fe/fe-core` for
LINEAGE). That single property flows
to:

- a maven-filtered resource inside the family's SPI module — what the
kernel expects;
- the `<manifestEntries>` of every jar built from that pom — what a
plugin declares.

Same build, same property, so the two can never disagree by accident;
across builds they disagree
exactly when they should. There is no second number, so no test has to
keep them in sync. All four
families start at `1.0`.

Families are derived, not enumerated: the token `connector` implies the
kernel resource
`/META-INF/doris/connector-plugin-api-version.properties` and the
manifest attribute
`Doris-Connector-Plugin-Api-Version`. A fifth family follows the
convention on both sides and needs
no new code in the gate.

**3. The gate**

`ApiVersionGate` (new, in `fe-extension-loader`, family-neutral)
compares **MAJOR only**. Minor and
patch are ignored in both directions, which is sound only because
"major" is defined as **any** change
to the SPI surface, additions included — a compatible minor can never
change the set of API elements a
plugin can reach. A jar that declares nothing is refused: fail-closed is
what makes the check
meaningful for plugins built before this existed.

`DirectoryPluginRuntimeManager#loadAll` takes the gate as a
**mandatory** fifth parameter, not an
optional one — an unwired gate would be the same silently-true check
this PR removes. It runs after
`loadClass` but before `asSubclass`/`newInstance`, so an incompatible
plugin's constructor never
executes and the operator gets a message naming both versions instead of
a `ClassCastException`.
Directory loading is the only production path and classpath
`ServiceLoader` builtins never pass
through it, so they are exempt structurally rather than by a flag.

All four families are wired. LINEAGE loads at start-up, and one refused
plugin still cannot stop FE.
AUTHENTICATION loads lazily and reports "no factory for this type" from
a different place than the
load, so the rejection reason is carried into that exception at both
call sites — otherwise a refused
plugin is indistinguishable from one that was never installed.

**4. Four SPI surface baselines**

One per family, freezing what that family's plugins implement or call —
including `fe-extension-spi`'s
`Plugin` / `PluginFactory` / `PluginContext` in all four, so a change
there turns all four red at
once. Signatures are recorded **with their return type**: a changed
return type is a major change by
the same definition, and the older name-and-parameters-only baseline
(`connector-metadata-methods.txt`)
cannot see it. No test can prove someone actually bumped the property —
a test sees the current state,
never the delta — so the failure message says, in the same commit as the
diff, that this is a major
change. The four baselines cannot carry a license header (every
non-blank line is read back as a
signature), so they join `connector-metadata-methods.txt` in
`.licenserc.yaml`.

**5. Two build facts, both found by trying rather than by reasoning**

- **maven-build-cache hashes a module from its sources, dependencies and
`<build><plugins>`, NOT from
`<properties>`** — and the filtered resource's source text is the
literal placeholder. AUTHENTICATION
and LINEAGE were going to ship with the property alone, since they have
no in-tree plugin zips, which
would have put their version outside every hashed input. Bumping 1.0 →
2.0 then produced an identical
module checksum (`4c1d34a6e04836c6` both times) and a cached jar still
declaring `api.version=1.0`.
All four families now carry the `<manifestEntries>` block, and the
checksum tracks the bump
(`a388a287b5a7e172` → `cfb1d83e91b92b98`). Worth recording how this was
caught: adversarial review
out-voted it 2:1 as "not a defect" on the reasoning that a major bump
always comes with an SPI surface
change — the one reviewer who actually ran the A/B build was the one who
was right.
- `fe-connector-es` and `fe-connector-trino` never excluded
`fe-filesystem-api` from their plugin zips,
shipping a duplicate that parent-first loading guarantees is never read.
Now excluded, so all eight
  connector zips agree.

Plan docs are updated to match: D-009 ("the API version never bumps
within this plan") is marked
superseded, and the master plan's §2.3 rule is replaced — the bump
criterion is now "any SPI surface
change, additions included", not "signature change or method removal".

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [x] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

Local verification, stated at the coverage it actually has:

- full reactor `package` with the **build cache disabled** and test
sources compiled
(`-Dmaven.build.cache.enabled=false`, no `-Dmaven.test.skip`): BUILD
SUCCESS. `test-compile` is not
sufficient here — `fe-connector-hms` compiles against a shade artifact
that binds to `package` — and
  the cache must be off for the reason in item 5.
- `checkstyle:check` on all six touched modules: 0 violations.
- tests: `fe-extension-loader` 19 (the gate's decision rule, and the
loader's behaviour on real plugin
jars assembled by the test with and without the attribute); `fe-core` 56
targeted (each family's
  wiring); authentication handler 20.
- the produced artifacts were read back rather than assumed:
`doris-fe-connector-es.jar` carries
`Doris-Connector-Plugin-Api-Version: 1.0` alongside the inherited
`Implementation-*` entries, and
`doris-fe-connector-spi.jar` carries `api.version=1.0` rather than an
unfiltered placeholder.
- **no e2e was run** (no cluster available here) and this PR adds no
regression case. The behaviour that
would want one — an operator's pre-existing plugin zip being refused —
is covered by the unit tests
  that build real jars, not by a deployment test.
- no BE file is touched by this PR, so BE was not rebuilt.

- Behavior changed:
    - [ ] No.
    - [x] Yes. <!-- Explain the behavior change -->

A directory-loaded plugin jar that declares no API version, or one whose
major differs from this FE's,
is now refused at load time; previously nothing was ever refused.
Fail-closed is deliberate — accepting
an undeclared version would leave the check meaningless for exactly the
plugins built before it existed
— and rebuilding the plugin against this tree is the fix. In-tree
plugins need no change: they inherit
the property from their family's parent pom, so the number appears in
exactly one place. Classpath
`ServiceLoader` builtins are unaffected, since they do not go through
directory loading.

- Does this need documentation?
    - [ ] No.
- [x] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

Not on the website yet, and not in this PR. Third-party plugin authors
now have to stamp one manifest
attribute, so this becomes user-facing documentation at the point where
external plugins are documented
at all — which this branch has not reached. Until then the runbook lives
with the design:
`plan-doc/designs/2026-07-29-plugin-api-version-check-design.md` covers
how to tell major from minor,
the Doris-side steps for each, what a third-party author must write, and
how to diagnose a plugin that
failed to load.

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
morningman added a commit that referenced this pull request Jul 29, 2026
… a manifest-carried API version gate for all four plugin families (#66211)

### What problem does this PR solve?

Issue Number: #65185

Problem Summary:

Part of the catalog-SPI migration (#65185). Review round 22, targeting
`branch-catalog-spi`.

One self-contained work line: give the four plugin families — CONNECTOR,
FILESYSTEM, AUTHENTICATION,
LINEAGE — an API version check that can actually reject a plugin. The
design, the version-change
runbook and the eight-task implementation plan are included under
`plan-doc/designs/`.

**1. The check that existed was true by construction**

`ConnectorProvider.apiVersion()` could never reject anything. The SPI
interface is listed
parent-first, every plugin zip excludes the SPI jar, and no shipped
connector overrides the method —
so the `default` body executed on a plugin's behalf is the kernel's own
code, and the number it
returns is the kernel talking to itself. Raising `CURRENT_API_VERSION`
and the default together
admits every stale plugin; raising only the constant rejects every
plugin, including freshly built
ones. Neither is a check. The other three families had none at all.
`apiVersion()`,
`CURRENT_API_VERSION` and the three comparisons against it are deleted.

**2. One number, physically distributed to both sides of the
comparison**

Each family declares its API version once, as a maven property in its
parent pom (`fe/fe-connector`,
`fe/fe-filesystem`, `fe/fe-authentication`, and `fe/fe-core` for
LINEAGE). That single property flows
to:

- a maven-filtered resource inside the family's SPI module — what the
kernel expects;
- the `<manifestEntries>` of every jar built from that pom — what a
plugin declares.

Same build, same property, so the two can never disagree by accident;
across builds they disagree
exactly when they should. There is no second number, so no test has to
keep them in sync. All four
families start at `1.0`.

Families are derived, not enumerated: the token `connector` implies the
kernel resource
`/META-INF/doris/connector-plugin-api-version.properties` and the
manifest attribute
`Doris-Connector-Plugin-Api-Version`. A fifth family follows the
convention on both sides and needs
no new code in the gate.

**3. The gate**

`ApiVersionGate` (new, in `fe-extension-loader`, family-neutral)
compares **MAJOR only**. Minor and
patch are ignored in both directions, which is sound only because
"major" is defined as **any** change
to the SPI surface, additions included — a compatible minor can never
change the set of API elements a
plugin can reach. A jar that declares nothing is refused: fail-closed is
what makes the check
meaningful for plugins built before this existed.

`DirectoryPluginRuntimeManager#loadAll` takes the gate as a
**mandatory** fifth parameter, not an
optional one — an unwired gate would be the same silently-true check
this PR removes. It runs after
`loadClass` but before `asSubclass`/`newInstance`, so an incompatible
plugin's constructor never
executes and the operator gets a message naming both versions instead of
a `ClassCastException`.
Directory loading is the only production path and classpath
`ServiceLoader` builtins never pass
through it, so they are exempt structurally rather than by a flag.

All four families are wired. LINEAGE loads at start-up, and one refused
plugin still cannot stop FE.
AUTHENTICATION loads lazily and reports "no factory for this type" from
a different place than the
load, so the rejection reason is carried into that exception at both
call sites — otherwise a refused
plugin is indistinguishable from one that was never installed.

**4. Four SPI surface baselines**

One per family, freezing what that family's plugins implement or call —
including `fe-extension-spi`'s
`Plugin` / `PluginFactory` / `PluginContext` in all four, so a change
there turns all four red at
once. Signatures are recorded **with their return type**: a changed
return type is a major change by
the same definition, and the older name-and-parameters-only baseline
(`connector-metadata-methods.txt`)
cannot see it. No test can prove someone actually bumped the property —
a test sees the current state,
never the delta — so the failure message says, in the same commit as the
diff, that this is a major
change. The four baselines cannot carry a license header (every
non-blank line is read back as a
signature), so they join `connector-metadata-methods.txt` in
`.licenserc.yaml`.

**5. Two build facts, both found by trying rather than by reasoning**

- **maven-build-cache hashes a module from its sources, dependencies and
`<build><plugins>`, NOT from
`<properties>`** — and the filtered resource's source text is the
literal placeholder. AUTHENTICATION
and LINEAGE were going to ship with the property alone, since they have
no in-tree plugin zips, which
would have put their version outside every hashed input. Bumping 1.0 →
2.0 then produced an identical
module checksum (`4c1d34a6e04836c6` both times) and a cached jar still
declaring `api.version=1.0`.
All four families now carry the `<manifestEntries>` block, and the
checksum tracks the bump
(`a388a287b5a7e172` → `cfb1d83e91b92b98`). Worth recording how this was
caught: adversarial review
out-voted it 2:1 as "not a defect" on the reasoning that a major bump
always comes with an SPI surface
change — the one reviewer who actually ran the A/B build was the one who
was right.
- `fe-connector-es` and `fe-connector-trino` never excluded
`fe-filesystem-api` from their plugin zips,
shipping a duplicate that parent-first loading guarantees is never read.
Now excluded, so all eight
  connector zips agree.

Plan docs are updated to match: D-009 ("the API version never bumps
within this plan") is marked
superseded, and the master plan's §2.3 rule is replaced — the bump
criterion is now "any SPI surface
change, additions included", not "signature change or method removal".

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [x] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

Local verification, stated at the coverage it actually has:

- full reactor `package` with the **build cache disabled** and test
sources compiled
(`-Dmaven.build.cache.enabled=false`, no `-Dmaven.test.skip`): BUILD
SUCCESS. `test-compile` is not
sufficient here — `fe-connector-hms` compiles against a shade artifact
that binds to `package` — and
  the cache must be off for the reason in item 5.
- `checkstyle:check` on all six touched modules: 0 violations.
- tests: `fe-extension-loader` 19 (the gate's decision rule, and the
loader's behaviour on real plugin
jars assembled by the test with and without the attribute); `fe-core` 56
targeted (each family's
  wiring); authentication handler 20.
- the produced artifacts were read back rather than assumed:
`doris-fe-connector-es.jar` carries
`Doris-Connector-Plugin-Api-Version: 1.0` alongside the inherited
`Implementation-*` entries, and
`doris-fe-connector-spi.jar` carries `api.version=1.0` rather than an
unfiltered placeholder.
- **no e2e was run** (no cluster available here) and this PR adds no
regression case. The behaviour that
would want one — an operator's pre-existing plugin zip being refused —
is covered by the unit tests
  that build real jars, not by a deployment test.
- no BE file is touched by this PR, so BE was not rebuilt.

- Behavior changed:
    - [ ] No.
    - [x] Yes. <!-- Explain the behavior change -->

A directory-loaded plugin jar that declares no API version, or one whose
major differs from this FE's,
is now refused at load time; previously nothing was ever refused.
Fail-closed is deliberate — accepting
an undeclared version would leave the check meaningless for exactly the
plugins built before it existed
— and rebuilding the plugin against this tree is the fix. In-tree
plugins need no change: they inherit
the property from their family's parent pom, so the number appears in
exactly one place. Classpath
`ServiceLoader` builtins are unaffected, since they do not go through
directory loading.

- Does this need documentation?
    - [ ] No.
- [x] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

Not on the website yet, and not in this PR. Third-party plugin authors
now have to stamp one manifest
attribute, so this becomes user-facing documentation at the point where
external plugins are documented
at all — which this branch has not reached. Until then the runbook lives
with the design:
`plan-doc/designs/2026-07-29-plugin-api-version-check-design.md` covers
how to tell major from minor,
the Doris-side steps for each, what a third-party author must write, and
how to diagnose a plugin that
failed to load.

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants