SPI class loader isolation

[karelkryda]

Hey @phax,
we are currently deploying your AP and we need to connect it to our Artemis MQ backend. We used forwarder SPI for it and also enriched it with few other functionalities needed for us. It mainly uses Artemis MQ client, S3 and DynamoDB (enhanced).
Since your AP supports S3 bultin (and few other low level deps), my question is how to properly "pack" the resulting SPI extension for compatibility.

Generally, fat jar (shadowJar) - with all deps for our SPI included - should be the right packaging mechanism, however I think the final AP runtime dependencies of yours and ours will clash.

To the question, could you please help us determine the right way to do it? How to properly pack it? How to properly start the AP to load our SPIs? Is there a way to have isolated class loaders?

Thank you in advance!

[phax]

Hi @karelkryda, thanks for the detailed write-up — this is a good question and it overlaps with another request we have (optional DB drivers), so we want to solve it generally rather than case-by-case.

Short answer: Don't try to merge your extension into our fat jar. Instead, we're adding a supported external extensions directory that gets added to the runtime classpath without rebuilding the AP jar. You drop your extension jar (and any deps we don't already ship) into that folder, and your SPI implementations are picked up automatically.

How it is supposed to work: The phoss-ap webapp is a Spring Boot executable jar. We're switching its launcher to Spring Boot's PropertiesLauncher, which lets you append external locations to the classpath via the LOADER_PATH environment variable (or -Dloader.path=...). Directories are scanned recursively for jars.

So the deployment model becomes:

LOADER_PATH=/ext java -jar phoss-ap-webapp.jar

or in Docker, mount a volume at /ext and set LOADER_PATH=/ext. Anything you drop into /ext joins the same classpath as the AP. Because our SPIs are plain ServiceLoader / META-INF/services lookups, your forwarder's META-INF/services registration is discovered with no extra wiring — just make sure your jar carries it.

How to package your extension (the important part): A full shadow/fat jar of your extension is exactly what causes the clashes you're worried about, because you'd be re-bundling copies of things the AP already has (Spring, phase4, ph-commons, the AWS SDK, etc.) on a flat classpath.

Instead, build a jar that contains only what the AP doesn't already ship:

• Mark every phoss-ap / Spring / phase4 / ph-* / shared dependency as provided (Maven) or compileOnly (Gradle) — you compile against them, but don't bundle them.

• Bundle only your genuinely-extra deps (Artemis MQ client, and whatever DynamoDB-enhanced / S3 pieces we don't already pull in).

• Align the versions of anything shared to whatever the AP ships, so you don't fight the classpath.

• To make that alignment easy, use the phoss-ap-parent-pom as Maven BOM, so you can import the provided versions and line up with ours automatically. (Note: we already ship the AWS SDK / S3 bits, so you likely only need to add the Artemis client — check before bundling.)

On isolated class loaders: To set expectations honestly - this gives you a shared external classpath, not per-extension child-first classloader isolation. In practice the provided-scope discipline above removes the version conflicts for the vast majority of cases, so we're going with this first because it's robust and simple. True classloader isolation (a plugin framework like PF4J, or JPMS module layers) is a much larger change — we're keeping it on the table as a follow-up only if a concrete clash can't be resolved by dependency alignment. If you hit one, please report the specific collision and we'll look at it.

We'll wire this up, test it, and add a "Deploying extensions / custom drivers" page to the Wiki. I'll follow up here once it's available.

[phax]

The extension demo submodule is out: https://github.com/phax/phoss-ap/tree/main/phoss-ap-extension-demo - Wiki page is work in progress

[phax]

And here's the Wiki page: https://github.com/phax/phoss-ap/wiki/Runtime-Extensions

[karelkryda]

Great, thanks!
I'll give it a shot tmrw and I'll let you know...

[phax]

Super - my local tests worked like a charme

[karelkryda]

Hey, we were able to start the AP with our SPI thanks to usage of BOM dependency handling and new ZIP based JAR loading you've done yesterday. I will proceed with Peppol TestBed cases now to verify everything works as expected.

During deployment verification, we observed the /api paths are not correctly covered by API token in our environment, opened the #62 for it!

Thanks so far, will let you know after testing is done on our side...