Plans for more core JRE classes?

[reportmill]

I'm interested in porting SnapKit UI toolkit to J2CL. I have previously done this with TeaVM and CheerpJ (see demos).

However, it seems while TeaVM has the same base JRE packages, it has significantly more coverage inside. This surprises me since J2CL seems to have significantly more support, activity, clients and contributors.

Question 1: Is J2CL intended for this use case (existing Java libs)?

Question 2: Is there a convention, accepted starting point, or "best-practices" for adding new JRE classes?

Here is TeaVM's class lib for comparison: TeaVM JRE. Some of the packages it includes currently missing from J2CL: java.lang.ref, java.lang.reflect, java.nio, java.text, java.net, java.util.jar, java.util.regex, java.util.zip.

This is a fantastic project, so I ask with admiration and respect! Thanks for any info.

jeff

[treblereel]

@reportmill if you are looking for nio, i have ported some classes to j2cl and i use it in j2cl quake2 port:
https://github.com/treblereel/gwt-nio

[reportmill]

Hi Dmitrii! Thanks for the response - this looks very nice!

Did you write this from scratch or start from a prior implementation? Also, is there any reason why this shouldn't be added to J2CL?

[treblereel]

@reportmill thanks

I relied on several projects like libgdx or Apache Harmony and gwt-nio is just a small part of what our community does.
J2CL team is consistently adding new features and emulations and if something is missed, it's not difficult to implement it by ourselves. There are many community projects that fill these gaps. So you can reuse such projects with bazel as the preferred way for Google or with j2cl-maven-plugin as i do. Maybe except for reflection api, but i am not sure, if we need it, mostly because it's too heavy and lack of it can be fixed by apt.

[reportmill]

I'm still surprised that the core JRE non-gui packages aren't a "must have" for most use cases. Is this because most J2CL projects are Android based? Otherwise it seems like a complete (as possible) JRE would be fundamental part of the MVP. I admit I'm from the Java desktop world, though. :-)

I don't rely on reflection for my browser deployments. But it's nice that TeaVM/CheerpJ support basic functionality, if only to make it easy to stub things out. I do think it leads to larger JS size if you use reflection in TeaVM.

[niloc132]

I think java.nio.* packages are a pretty good example of this case, where "simply" emulating them might have surprising results. For example, the project that @treblereel linked is a very complete implementation of those packages, whereas https://github.com/Vertispan/gwt-nio/ is a deliberately incomplete version (sources copied, updated from the PlayN project). The incompleteness stems from limitations in the browser - while @treblereel sought complete emulation to allow a project to be entirely agnostic of running in the JVM or browser (...from the perspective of the java.nio packages at least), my goal was quite different, to allow an application to use the browser TypedArray APIs, yet still run on a normal JVM.

Some quick examples:

• ByteBuffer.wrap(byte[]) - this API ostensibly takes a Java byte[] and turns it into a ByteBuffer, and gives certain expectations about performance. On the JVM for example, while this will still use a java byte[] on the heap, it will access the data using unsafe operations - put(ByteBuffer)/put(byte[]) will directly copy memory, rather than loop through the bytes in the array. In J2CL/GWT however, byte[] (and other java arrays) are presently implemented through JavaScript arrays - items may or may not be contiguous, a "byte" will likely take several bytes to represent, and it is unlikely that there are efficient batch operations to copy large amounts of them at a time. Nevertheless, to be complete and allow the application the illusion of running on a complete JVM, https://github.com/treblereel/gwt-nio implements this method, with the resulting ByteBuffer being a wrapper around a JS array. On the other hand, https://github.com/Vertispan/gwt-nio/ does not offer this method, but instead requires that memory be allocated first through the TypedArray api, or just through ByteBuffer.allocate (or ByteBuffer.allocateDirect, even thought it won't actually be able to produce off-heap memory), to ensure that the ByteBuffer won't be backed by anything other than a TypedArray.
• LongBuffer can be emulated of course, and indeed that is what a complete implementation will do, but as of the last time I was working on https://github.com/Vertispan/gwt-nio/ there wasn't a BigInt64Array to work with (and with that said, I'm not sure there is a clean way to treat JS BigInt as a Java type as it isn't really a long, but has arbitrary precision). This means that the LongBuffer would either need to wrap long[] and thus store many objects (j2cl uses goog.math.Long to implement Java long) in a JS array, or would store 8 bytes and decode a long each time one is read (faster copying or translating to/from ByteBuffer etc, but still slow reads and writes).

This seems to me to be a point in which reasonable people can disagree - do you make the emulation as complete as possible, with potential pitfalls, or do you make the API clearly something that makes sense to run in a browser?

[reportmill]

Thanks Colin - those are very insightful observations and gave me a lot to consider!

However, I would say the clear choice is make the J2CL JRE match the functionality of the standard JRE as much as possible despite drawbacks, for several reasons. For one: this would lead to more out-of-the-box compatibility. And two: As J2CL transitions to WebAssembly, many of these mismatches with JavaScript may go away.

I think most developers are happy to have standard library functionality even if it eventually falls short in cases. Those cases can be addressed with customizations and optimizations to the app and/or the JRE in successive versions.

I am certainly thrilled with the TeaVM JRE implementation, even though it also is incomplete. My apps aren't as functional in the browser (yet), but deployment is a huge win on that platform. Users that eventually demand more can move to a desktop version.

[gkdn]

In general we are very conservative in what we provide as emulation (with the exception of some legacy choices that we inherited from GWT).

The reasoning behind that is; J2CL targets code bases where thousand of engineers are working on products with huge dependency graph and not necessary web experts themselves. In order to be able to scale at that level; we try to minimize anything surprising related performance, code size and correctness.

This strategy has been working well for us but I do understand that it may not for all. The biggest negative is; less libraries work out of box if you just want to able to just compile something which wasn't a priority for us. In our case we would rather break during compilation and have the user either:
a- Propose a change the 3rd party library code itself.
b- Avoid the dependency to the library in the first place.
c- Explicitly opt-in by pulling 3rd party emulation of the missing JRE.

[gkdn]

Also see: https://github.com/google/j2cl/blob/master/docs/limitations.md

[reportmill]

Thanks Goktug - that makes good sense. I will have to revisit this later this year when I can invest more time on missing dependencies for SnapKit. If anyone else is struggling with similar use case or similar need for more JRE, please leave a comment!

[treblereel]

@reportmill there is an active community https://gitter.im/vertispan/j2cl of j2cl users who are focused on libs and frameworks around it (we mostly use maven, but it's not an issue). So lets join forces!

As for your initial question, we can write ( and we do) enumations for missed classed by ourselves, but ppl from outside google struggle with issues like 'java 11 support and above', better enums support and so on (if forces us to support a fork). According to J2CL Now & Next at least 'java 11' is pretty close to be released coz bazel5 has been released as well.

[gkdn]

I already switched to new bazel and enabled the Java 11 support for open-source; I haven't updated the list yet.

What are you referring to as better enums support? Enum.valueOf?

[treblereel]

yeah, Enum.valueOf would be nice, plus @JsEnum

i working on webgpu externs ( well, it's mostly webidl parser) to generate elemental2-like wrappers, so for now i have to use string instead of enum . But as far as i remember, it's more closure issue. But i can be wrong

I really appreciate your help

[gkdn]

Enum.valueOf is a reflective API so we cannot realistically support that as it would cause code bloat. We were only considering to support very limited scenarios, e.g. when the parameter is statically known, however that won't help a lot use cases when you can simple do FooEnum.valueOf instead.

For JsEnum, are you referring to limitations like MyJsEnum[]?