Extension Structure ADR proposal

[cescoffier]

First draft (following #47074)

Ping: @gsmet @geoand @dmlloyd

[dmlloyd]

One problem I think we have is that non-extension packages are in a conflicting namespace with extension packages. Would it be possible to change the recommendation going forward to be something like io.<quarkus|quarkiverse>.ext.<extension-name>...? Otherwise, we could put all non-extension classes under a single subpackage to minimize the conflict e.g. io.quarkus.core.xxx. Either way is breaking stuff but would prevent more/worse breakage in the future.

[cescoffier]

I would go with the latter.

Vert.x 2 and 3 (at the beginning) tried to use ext packages - it didn't fly. Very quickly, the ext looked odd and inconvenient.

[dmlloyd]

In that case we have a lot of packages to rename.

[aloubyansky]

What about org.apache.camel.quarkus:camel-quarkus-activemq? Should we make the examples more generic?

[geoand]

I will have a look tomorrow

[geoand]

Isn't the runtime module necessary, if for nothign else but to contain META-INF/quarkus-extension.yaml?

[cescoffier]

Not really, we have extensions that are not "true" extension, in the sense they don't have a runtime:

For example:

• jms-spi, which only contains a deployment module (that should be a deployment-spi module with the new nomenclature)

We could consider that extension as malformed.

[geoand]

I wasn't aware of that one, indeed it seems weird

[geoand]

This sentence needs to be improved. Are consumers meant to be other extensions in this context or applications? Also, I assume you mean avoid being tightly-coupled

[cescoffier]

I think I need a section about the coupling. It's not clear (it is in my head, but not in the document, and the graph is not rendered in the preview)

[cescoffier]

Yesterday, I drafted that proposal, and I gave some thought about it last evening (yes, I should have considered doing this in the opposite order).

I've two questions:

• JPMS name: Should only the runtime module have a JPMS name (and maybe spi), or should every module have one, as proposed? (@dmlloyd)
• In the case of an extension having no public API in its runtime module and having its public API in the spi module, what should be the root package of the spi module: io.<quarkus"quarkiverse>.<extension-name>.spi like proposed or just io.<quarkus"quarkiverse>.<extension-name>. The latter can introduce a split package risk if, at some point, the runtime module starts adding a public API.

[holly-cummins]

Related (on the linting side): quarkiverse/quarkiverse-devops#231

[dmlloyd]

• JPMS name: Should only the runtime module have a JPMS name (and maybe spi), or should every module have one, as proposed? (@dmlloyd)

IMO yes (every module should have a JPMS name) - we will have to tackle this eventually and it's better to have it done ahead of time. Perhaps our tooling can set this up automatically?

• In the case of an extension having no public API in its runtime module and having its public API in the spi module, what should be the root package of the spi module: io.<quarkus"quarkiverse>.<extension-name>.spi like proposed or just io.<quarkus"quarkiverse>.<extension-name>. The latter can introduce a split package risk if, at some point, the runtime module starts adding a public API.

I think spi, but I also think that the distinction between API vs SPI needs to be made crystal clear in the docs.

[Ladicek]

The spi module that contains runtime SPI seems backwards. Most extensions do not expose public API directly, they just expose the API of the underlying library/framework. But I know we do have extensions that have public API (Redis Client comes to mind).

In my personal opinion, public API of an extension should be either in the runtime module, under io.quarkus|quarkiverse.<extension>, or in a separate module (called api, not spi), in the same package.

The runtime part of the extension, which is not a public API, should always be in the runtime module under io.quarkus|quarkiverse.<extension>.runtime.

I see no reason to expose runtime SPI in a different manner than runtime API. They should both be in the same place. If the runtime API has a reason to delimit an SPI, it can do it in all the traditional manners -- we don't need an extra module in the extension just for a runtime SPI. If we need outside world to be able to depend on a runtime API of an extension without depending on the extension as a whole, the full runtime API of the extension should be in a separate module (again, called api, not spi).

My $0.017.

[cescoffier]

@Ladicek I think it's mostly a naming issue (what is not a naming issue these days 😄).

Let me try to clarify how we’re thinking about the separation between runtime, spi, and potential api modules.

API vs. SPI

It’s important to distinguish two concepts (as @dmlloyd said). Here is my attempt:

• API: Types (classes, interfaces) intended to be used by application code or other extensions to consume functionality. Think: RedisClient, annotations, utility methods, etc.
• SPI: Types that allow others to contribute to the extension’s behavior, typically via user implementations, configuration classes (customizer), or extension points. Think: custom resolvers, listeners, info contributor, etc.

We agree here: API is about usage, SPI is about participation.

The Coupling Dimension

(Not this is very badly explained in the draft as @geoand mentioned)

Where things get more subtle is around coupling: does the application (or consuming extension) require the extension to be present at runtime?

• Strongly coupled: If you use an API, like RedisClient, you clearly need the Redis extension present. This API can safely live in the runtime module under the root package.
• Loosely coupled: But if you’re contributing something optional (e.g., a type annotated for discovery, or consuming a CDI event), the extension might not need to be present. That’s where we want to avoid pulling in runtime transitively, why we have these SPIs into a spi module.

In other words, this separation is about allowing contributions without forcing runtime presence, more architectural than semantic (and that's my whole problem...).

Why not call it api?

You suggested that the public API should live in a module called api, not spi. It makes sense.

If we were trying to reserve api for consumption (i.e., the user-facing surface) and spi for contribution (i.e., extension points), it makes sense to clearly isolate both. The info extension does that very well.

• Should the spi module be named api (and contain both SPI and API in different packages)? In this case we will force the coupling.
• Should we have 2+1 separated modules (spi, api, runtime), it's a valid suggestion. Do we need that extra complexity ?

That's definitely open for debate and suggestions.

What about consumed CDI events?

That’s, for me, a great case where things blur: the application can consume events produced by an extension without needing that extension to be present. It feels like a runtime API, but from a dependency standpoint, it behaves more like an SPI (you can consume it, even if the source isn’t there).

In those cases, having the event types in the spi module makes sense, not because they are technically SPIs, but because we want to avoid runtime coupling.

What about?

• Public API used at runtime → runtime module, root package
• Extension points and loosely-coupled contribution interfaces → spi module, .spi subpackage
• Split based both on intent (API vs SPI) and coupling (runtime presence vs isolation)

WDYT?

P.S.: Realizing that this comment is almost as long as the initial ADR...

[geoand]

API is about usage, SPI is about participation.

+1

In other words, this separation is about allowing contributions without forcing runtime presence, more architectural than semantic (and that's my whole problem...).

+1

[aloubyansky]

For the sake of consistency with spi, which implies it's runtime spi, should it just be dev?

[aloubyansky]

I was also thinking about adding codestart.

[cescoffier]

Oh yes, forgot codestart.

[aloubyansky]

deployment does not need to depend on runtime-dev. It may be necessary but there is no general reason for that.

[aloubyansky]

runtime may depend on runtime-dev. And runtime-dev may appear to depend on runtime. It really depends on a specific case.

[aloubyansky]

In our current codebase, in terms of Maven dependencies, runtime-dev depends on runtime, which is what's important for a project setup for an extension developer.

In the Quarkus dependency model (which is a hidden magic) runtime appears to depend and pull in runtime-dev in dev mode.

It doesn't mean other users have to use dev-mode dependencies in this specific way though.

[Ladicek]

So in my eyes, SPI is part of API. API is what users are supposed to use. Some usages are consumption, some are participation, but it's all API. If you feel like there's a clear line between consumption and participation, I'll respectfully disagree; outside the most simple cases, participation often requires consumption. In other words, an SPI typically looks like

interface Participant {
    void doSomething(Context ctx);
}

where Context either is part of API directly, or provides access to parts of API indirectly. An attempt to separate the two into different modules is an exercise is futility.

[cescoffier]

@Ladicek We have an example of clean delimitation (like the info extension). However, let's imagine we can't have a clean cut (and the CDI event is an example of this ambiguity). Should we have an api module that contains both the API and SPI, but without binding the consumer to the extension? The consumer can decide to either force the extension to be present (by depending on the runtime module) or keep the coupling loose (by depending only on the API module).

[Ladicek]

However, let's imagine we can't have a clean cut (and the CDI event is an example of this ambiguity). Should we have an api module that contains both the API and SPI, but without binding the consumer to the extension?

Yes indeed, that's what I'm advocating for.

[aloubyansky]

It seems like a split of api(spi)/runtime could be justified if there could be multiple implementations of that API/SPI, i.e. different extensions implementing them. Otherwise, bundling it all in a single runtime module shouldn't be a big deal, should it?

[cescoffier]

Otherwise, bundling it all in a single runtime module shouldn't be a big deal, should it?

If we add the classes into the runtime module, we end up having a tight coupling. If we add them into an api module, yes, it should not be a big deal. The issue is not the split, it's the coupling.

[aloubyansky]

I may have missed it, what's the problem with coupling in cases when there is only one impl of a given API?

[cescoffier]

@aloubyansky it's mostly what I tried to explained here:

---

The Coupling Dimension

(Not this is very badly explained in the draft as @geoand mentioned)

Where things get more subtle is around coupling: does the application (or consuming extension) require the extension to be present at runtime?

• Strongly coupled: If you use an API, like RedisClient, you clearly need the Redis extension present. This API can safely live in the runtime module under the root package.
• Loosely coupled: But if you’re contributing something optional (e.g., a type annotated for discovery, or consuming a CDI event), the extension might not need to be present. That’s where we want to avoid pulling in runtime transitively, why we have these SPIs into a spi module.

In other words, this separation is about allowing contributions without forcing runtime presence, more architectural than semantic (and that's my whole problem...).

---

If we put API+SPI in the runtime module, consumers will be tightly coupled: the extension will be pulled.
If we put API+SPI in an api module, consumers can decide their coupling:

• depending on the api module will not pull the extension
• depending on the runtime module will pull the extension (the runtime module depends on the api module)

[aloubyansky]

Thanks. So strong coupling is still an extension developer choice.