Scala Language Plugin for Pulumi should support all the executors that Java Language Plugin does

[dubinsky]

Getting started creates an impression that Scala CLI and Scala Language Plugin are required to use Besom. In fact, if the project uses one of the execution methods supported by the Pulumi Java Language Plugin (Gradle, Maven, JAR, JBang or SBT), Besom SDK can be used with runtime: java in Pulumi.yaml, just as Pulumi Java SDK can be used from Scala code that way. Indeed, if the project uses Gradle, Maven or JBang, Scala Language Plugin must not be used: while retaining support for JAR and SBT execution, it drops support for Gradle etc. in favour of Scala CLI. Scala Language Plugin seems to be required only if the project uses Scala CLI, and although this setup is recommended, I think the ability to use Besom with - say - Gradle should be at least mentioned ;)

[lbialy]

That is definitely true (in fact, before we've had language plugin we used runtime: java). I see no reason not to document that and I didn't include that fact in the first draft of docs only because I strived for a lean, uncluttered and frictionless experience (as all DX should be!). I think we can add sections regarding the use of other build tools available on the platform. In fact, we should also support mill on Scala language plugin's side if we don't do it yet.

…

On Thu 16. Nov 2023 at 21:06, Leonid Dubinsky ***@***.***> wrote: Getting started <https://virtuslab.github.io/besom/docs/getting_started/> creates an impression that Scala CLI and Scala Language Plugin are required to use Besom. In fact, if the project uses one of the execution methods supported by the Pulumi Java Language Plugin (Gradle, Maven, JAR, JBang or SBT), Besom SDK can be used with runtime: java in Pulumi.yaml, just as Pulumi Java SDK can be used from Scala code that way. Indeed, if the project uses Gradle, Maven or JBang, Scala Language Plugin must *not* be used: while retaining support for JAR and SBT execution, it drops support for Gradle etc. in favour of Scala CLI. Scala Language Plugin seems to be required only if the project uses Scala CLI, and although this setup is recommended, I think the ability to use Besom with - say - Gradle should be at least mentioned ;) — Reply to this email directly, view it on GitHub <#303>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACBVNUR5MS4ZVYZVNOMJCI3YEZW2LAVCNFSM6AAAAAA7OXAXUWVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE4TONRXGY4DKMQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[dubinsky]

@lbialy one way to remove some friction is to copy from the Java language plugin to the Scala one not just the JAR and SBT executors but all of them (Gradle, Maven and JBang too); this way, by switching from the Java runtime to the Scala one one would gain the ability to use Scala CLI - but would not lose the ability to use - say - Gradle...

[dubinsky]

I am not sure what the Scala language plugin does, but I suspect that it provides more than just the executors. I can easily see how Besom might require the use of the Scala language plugin, and I am more than willing to switch to runtime: scala - as long as I do not lose the ability to use the build tool of my choosing that I can use now with runtime: java.

[lbialy]

Yeah, so the long game plan for language plugin is to support way more features as planned by Pulumi, programmatic code generation on demand on user's machine for starters. Current impl is just the same go app as pulumi-java with changes so we could backport gradle etc very easily I think. In the long run we want to rewrite it to Scala and compile with native-image.

…

On Thu 16. Nov 2023 at 21:33, Leonid Dubinsky ***@***.***> wrote: I am not sure what the Scala language plugin does, but I suspect that it provides more than just the executors. I can easily see how Besom might require the use of the Scala language plugin, and I am more than willing to switch to runtime: scala - as long as I do not lose the ability to use the build tool of my choosing that I can use now with runtime: java. — Reply to this email directly, view it on GitHub <#303 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACBVNUU4IQDJLI7DIMRL6GTYEZ2CHAVCNFSM6AAAAAA7OXAXUWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJVGI3TAMRYGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dubinsky]

plan for language plugin is to support way more
features

Cool!

we could backport gradle etc very easily

From what I see, JAR and SBT support were just copied, so it's just more copying ;)
Ability to switch to runtime: scala without breaking Gradle is going to greatly improve DX ;)

[pawelprazak]

The fact that those language plugins are mostly interchangeable right now is not intentional.

The Java SDK and it's language plugin have different lifecycle and ownership - it is maintained by Pulumi
The Scala SDK a.k.a. Besom is maintained by VirtusLab.

This is the first reason why we cannot guarantee the interchangeability of language plugins in the future.

The second, and more important fact is, that the language plugin has more responsibilities then just build tools.
Please have a look at the gRPC definition for reference.
This is not implemented in both mentioned plugins at the moment of writing, but it will be implemented in Besom, rather sooner than later.
This will make the interchangeability impossible by design.

Another internal goal of Besom is to rewrite the language plugin to Scala 3 with Graalvm static binary, this will allow us to embed the codegen module and maybe even the scala-cli itself, it sounds tempting.

As for the tooling support, I've always thought the Pulumi way of doing language plugins is fundamentally incompatible with the "JVM-way", and the dependency should be IMO reversed.

Once we have Automation API implemented, every build tool can easily create a plugin for Besom, just like most JVM people like it (it was highly requested by Java SDK users to have a gradle and maven plugins).

So to summarize, I'm in favor of the following plan:

• implement the Automation API (it is critical anyway) [Epic] Add Automation API #254
• drop SBT support and let the community develop plugins leveraging Automation API
• port language provider to Scala and integrate it with coursier/scala-cli as a single binary

Also we'd like to prioritize delivery of feature parity with upstream - this is the main goal, so we must be laser focused on the core of Besom.

[pawelprazak]

As for something immediately actionable, we probably should document that Java SDK and Besom are completely separate and can interact in unpredictable and "interesting" ways when used together.

[dubinsky]

@pawelprazak
I understand that to continue to be able to use Besom I need to use runtime: scala, and I am happy - eager even! - to do it.
I also understand that when the Automation API becomes available, build-tool integration using it will become possible, and I am happy to look at the possibility of developing a Gradle plugin for such integration.

I do not quite understand the impact of this statement:

I've always thought the Pulumi way of doing language plugins is fundamentally incompatible with the "JVM-way", and the dependency should be IMO reversed.

Does this mean that at some point the ability to use Besom via the pulumi CLI delegating to the Besom's Scala Language Plugin is going to get removed completely?

Before all of the above transpires, what is the way for me right now to use Besom properly, with the Scala Language Plugin, if I am using Gradle as a build tool and the language plugin does not (and will not) support it? Should I point to my executable JAR in Pulumi.yaml's runtime.option.binary or is there a better way? Or am I out of luck and not a part of the target audience of the early adopters? ;)

Thanks!

[pawelprazak]

I'm sorry about my confusing remarks, I was just thinking out loud. There are no specific decisions made yet regarding language plugin, other than that we'd like to find a way to give the community the best support for build tools possible, and the exact shape of it, I hope, will be made clear by user feedback, like yours :)

This is a very good point you are making, about early adoption, and I see no reason no to port Java lang host Gradle support, @lbialy WDYT?

[dubinsky]

Will this be in the next release? ;)

[golosegor]

it would be really nice to have this resolved...

[lbialy]

Ok, I'll take a look at this. I think we can include this in the upcoming 0.2.0.

…

On Wed 3. Jan 2024 at 18:37, Egor Goloshchapov ***@***.***> wrote: it would be really nice to have this resolved... — Reply to this email directly, view it on GitHub <#303 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACBVNUUQLS3GCCGUUFXZVGDYMWJONAVCNFSM6AAAAAA7OXAXUWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZVG4ZTCMZZGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[pawelprazak]

Reopening until release

[lbialy]

Release of 0.2.0 which brings support for Maven and Gradle is underway. Closing the issue.