Native Scala.js and Scala Native (later) integration (so separate %%% won't be needed)

[keynmol]

Feel free to close if this has already been discussed to death, but please link to the discussion.

---

Proposal

Natively integrate Scala.js and Scala Native (post-1.x) projects into SBT, as part of its metamodel, along with the syntax for cross-platform dependencies

Context

While Scala.js is used in production and a lot of libraries cross-build for it, it's availability as a plugin only makes it difficult to

1. Manage dependencies safely:

   • %%% vs %% is a constant pain point and can lead to downstream breakages. If it were supported natively in SBT model, build tool would be able to provide rich diagnostics to the user, and detect binary incompatibilities between different SJS/SN versions
   • Unification could simplify setup of projects and libraries by users familiar with SBT but not familiar with SJS and sbt-platform-deps syntax

2. Cross-build for multiple SJS versions

   • Currently this has to be outside of SBT (common solution is an env variable).
   • Might be irrelevant if there never will be two different adopted lineages of SJS (like 0.6 and 1.x were), but it's not unlikely that Scala.js 2.x will introduce the same problem

3. Integrate/interact with the build tool

   • Things like Bloop and project matrix lookup the plugin and Scala.js keys via the use of reflection (see example in bloop)
   • If Scala.js configuration was an intrinsic part of SBT's project metamodel, those integration could be more precise

4. ??? other reasons

Why

Scala.js is an integral part of Scala ecosystem and, for example, recognised as such in Mill, and it would be excellent if SBT was to integrate deeper with it

How

I have no idea at this point. Let's discuss.

---

Caveat: Turns out I'm not so good at writing proposals, so I'm hoping to gain feedback from people who actually maintain libraries that crossbuild for SJS.

Interested to hear from @sjrd and @eed3si9n on this.

Sparked by a conversation on Discord with @tpolecat @armanbilge @mpilquist @Daenyth and @SethTisue

[Daenyth]

A specific note on %%%. If it was part of sbt, then library and sbt documentation could suggest using it in all places where users currently have %%, removing a source of confusion for new learners and streamlining the process for everyone.

Arman was also mentioning that this is a source of breakage;

what's even more scary is suppose you start your project JVM with %%
then you add a SJS build, but forget to change %% to %%%
you can make it as far as publishing artifacts without realizing something weird is going on
like, until downstreams complain, then you realize 🙂
true story btw

[keynmol]

I can only guess that the pom.xml for those _sjs1 artifacts was referencing artifacts without _sjs1 and that's why you got linking errors.

Publishing was unaffected.

[armanbilge]

https://repo1.maven.org/maven2/org/typelevel/cats-effect-testing-scalatest_sjs1_2.13/1.1.1/cats-effect-testing-scalatest_sjs1_2.13-1.1.1.pom

[keynmol]

Seems like it's referencing a JVM version of scalatest

[armanbilge]

I think the cats-effect-testing SJSIR was referencing methods that existed in the cats-effect JVM artifacts but not in the cats-effect SJSIR artifacts. So even though my downstream properly had cats-effect SJS dependency, it was essentially like a binary-incompatibility.

[armanbilge]

Seems like it's referencing a JVM version of scalatest

True, but I also had a SJS dependency on that in my downstream. The linking errors were on cats-effect APIs.

[eed3si9n]

%%% vs %% is a constant pain point and can lead to downstream breakages.

If we were to in-source Scala.JS support, what I suggest would be to obliterate %%% and do whatever it needs to do (capture sjs prefix under JS subproject etc) as part of normal %%/sbt operation.

[keynmol]

Agreed! The benefit of just %% is that old stuff can stay working and %%% can be deprecated in the plugin.

%% also works better when extracting dependencies into a separate object/val, with %%% I think you need to use Def.setting {... %%% .. % ...}

[eed3si9n]

To add some details, I am guessing that sometimes we do want plain %% JVM dependencies in SJS subproject for compile-time dependency such as scala-library.jar so if we unify %% and %%%, then ModuleID or cross would have to be aware of the notion of prefix as:

• None / automatic - which sbt would infer as "sjs"
• Some("") - intentional JVM dep on SJS
• Some("foo") - something custom

[sjrd]

There are three major difficulties that any serious proposal for this will have to address:

• How to make this happen without breaking the binary compatibility of sbt-scalajs and sbt-scalanative.
• How not to force Scala.js and sbt into having a circular dependency on each other. The build of Scala.js defines sbt-scalajs and uses it for its own purposes including testing everything. For that, it needs to depend on sbt; how does sbt then depend on Scala.js?
• New versions of Scala.js introduce new features into sbt-scalajs, sometimes in close collaboration (for new linking features notably, for example recently for module splitting). How can Scala.js introduce new linker-level features if it does not control sbt-scalajs? (this is related to the previous point, since Scala.js needs a way to test the features it introduces)

[armanbilge]

That looks fantastic too :) if sbt 1.x can get some built-in cross-platform capability that would be wonderful.

[nafg]

Not sure if this is the same or different, but one angle is that SBT could introduce a built in notion of platform, or of N versioned artifact suffix appending things, in a way that concrete platforms would have a static API for builds to use, while SBT being open to plugins defining platforms etc in a way that might be dynamic under the hood.

The reason I mentioned more than just platforms is that another use case is other "meta version" matrices. An example that comes to mind is flyway-play and other libraries that need their artifacts to be matched to the version of some other thing, while also needing an artifact version so they can enhance whatever it is they're providing.

[eed3si9n]

@nafg sbt-projectmatrix does support custom axis that can represent arbitrary library variation - readme uses Typesafe Config 1.2 vs 1.3.

However, this is slightly different concept from the platform prefix. You can think of platform prefix to be an implicit parameter that is applied to (almost) all ModuleID instances, like JVM (blank) vs JS 0.6 vs JS1 vs Native.

[nafg]

(why I said not sure if this is the same or different ;) )

[nafg]

(could be taken as a more limited proposal I guess)

Question - I like sbt-projectmatrix (I used it in https://github.com/nafg/mergify-yaml/), but I think I heard (maybe Sébastien said something to that effect IIRC) that it uses more memory than crossproject. If it were made part of SBT would that be different? (Also IIRC I had an issue with it in IntelliJ, hopefully that would be fixed too...) Aside from that (but especially if it improves performance) I think it's a good idea.

[armanbilge]

Anton has just escalated: https://youforgotapercentagesignoracolon.com/

sbt, your move 😝

[SethTisue]

for those who cantrememberareallylongurl, https://sca.la/percent

[armanbilge]

I opened an issue to try and action a (small) step forward on this.

• Upstream sbt-platform-deps into sbt #7057