Code direction

[Thrameos]

I would like to check which enhancement is most valuable to the current user base now that I am back online.

Here are the options:

• webfx - a full standalone gui environment that unified marshalling between Javascript and Python. This makes async calls from Python to Javascript and the opposite a first class citizen. Should include support for angular.
• reverse bridge - I completed this a year ago, but it required a new test harness. Advances in ai mean I may be able to pull it off without user support.
• unified gc - I prototype it a year ago and finally am getting permission to work towards a publication. It still requires a formal proof.
• memory model - I wrote a complete in memory model for jpype that abandons the allocator that is so problematic. But never received a blessing from anyone in cpython team to push it through. My only hesitation is that requires a different mechanism for different types (simple, variable int, exception)
• string - this one requires cooperation fron cpython team. But it would remove our largest friction point.

Technically I could pull all off in a year, but that would require assistance from a cpython team so I likely will be restricted to choose two. Webfx is the only one with internal funding.

[Thrameos]

@marscher @pelson I separate issue that I am tracking for future directions is the impending downgrading of JNI. As anyone should be aware JNI is being phased out in favor for panama, but panama is mostly geared to downcasting. I was on travel all week so I had a fair about of time waiting in airports or stuck in hotels, so I tried building a JNI replacement with panama. Unfortunately, if I make such a substitution it means we will end up stepping up the Java requirements all the way to Java 22.

I am thinking the only way to do this properly would be to fork the project and use the fact that our project is named "jpype1" in pypi. We can make post JNI version called JPype2 and avoid the horrible nightmare that would come from having two backend models. The upside of this is as the new protocol is transaction based, we would be able to support shared memory or even remote JVM models rather than pure in process.

Any thoughts on project directions?

[Thrameos]

I put in a solid weekend on the Panama "Java Second Native Interface" project. It is shaping up great because it will actually speed up the array bulk transfers even more than we currently have, but until I get a full version running and benchmarked I don't know if it is viable. I will likely just push it to my personal github next weekend, then we can decide what to do.

[marscher]

I do like the reverse bridge and the new Panama thing sounds interesting as well. If you're going towards the Panama direction, I also recommend a second branch if the models are so fundamentally different.

[ryanmkurtz]

The reverse bridge would be of interest to Ghidra. Right now users have to launch Ghidra a special way if they want it to be PyGhidra-enabled, which is undesireable. Sounds like with reverse bridge they could just launch Ghidra as a normal Java process like normal, and potentially have native CPython support available.

[Thrameos]

Okay I will see about switching back to the reverse bridge work. I need to ship the 1.7.1 release. My Panama experiments are going okay so far in that I have a code base with about 90% working in the JNI side, but still too many rough edges to show. Ais have improved greatly on the test generation part so that reverse bridge will likely progress. My hang up there was exception handling (always a rough spot), testing, and then the big one how to pass a PyObject to Java. Currently we just block it entirely so we aren’t likely to break much, but if allow implicit conversion it will have broad implications. The safest will be require ***@***.***)” where PyObject is something one imports from “python.lang” which is the Java entity definitions.

[Thrameos]

I completed the first prototype of the J2NI support library late last night and submitted it for Review and Release. Based on the size of the package and its relatively low-risk profile as general-purpose shared software, I expect the review cycle may take on the order of 30 days. I will need to wait for that process to complete before posting it and beginning the next phase of JPype integration work. At this point, I would characterize the project as being in a late prototype phase. At a high level, the library is built around a POD-oriented transport model, with references represented as int64_t values. It includes an ABI installer that dynamically links Java-provided symbols, and once initialized it provides the core mechanisms for class, field, method, and local/global object references. Traditional pushFrame and popFrame semantics are preserved. One of the main architectural changes relative to JNI is that array access is no longer based on shared or pinned GC-managed memory. Instead, the design uses explicit view-based transfer operations. Because the transport is POD-oriented, the model works naturally in-process and could potentially support other transport modes in the future. I also moved a substantial portion of the common runtime support needed for JPype into the base library. That should reduce the amount of JPype-specific bookkeeping required during bootstrap and steady-state operation, particularly around reference handling and other runtime utilities. Proxy and Cleaner support are also included in the core library, which should help keep the JPype-specific layer smaller and more focused. From an architectural standpoint, the design reduces several categories of cross-boundary bookkeeping by using a shared context, or mailbox model, for status, exceptions, arguments, and results. The main open question is the cost of Java-side marshalling, since this approach shifts a significant amount of bridge behavior into Java. That remains the main area that still needs measurement. Current test coverage is approximately 90% of branches, with 760 Java tests completed so far, and I expect a comparable amount of C++ testing once that side is exercised more fully. The Java test suite is currently passing on Java 25. The next step is to validate the C++ layer and begin benchmarking. The main unresolved areas are monitors, launching, and thread management. At present, Panama does not appear to provide enough support in those areas to fully eliminate JNI dependencies for all runtime-critical functions, which remains an important limitation for this type of migration. I also added structured marshalling and unmarshalling support for larger argument lists and bulk-style transfer patterns, which may be useful for higher-volume operations such as database-oriented unpacking and similar workloads. I included both C and C++ bindings, and the design may be adaptable to other languages such as Rust.

[Thrameos]

I moved back to the reverse bridge. Getting it synced with the current presented little difficulty but JPype still needs a series of core upgrades before this concept can be reviewed or it is going to be another one of those too many things at once.

In particular we have the ghost context pointer that is floating around everywhere to cleanup. I was not able to maintain it in the reverse bridge as it was leaking into downfalls everywhere.

Second the proxy class is way too heavy currently. It should be doing all is work eagerly on the first proxy request so that the hot path is clean and lean. These two changes can be pulled out of the reverse bridge and applied separately as 1.8 before we get to the reverse wrapper. It should moderately speed stream api calls from Python.

[Thrameos]

@marscher Do you have any preference how to handle these PR? My thinking is we should make a development branch (devel) first which starts the current head, then apply a clang beautifier to the code so that we are only looking at at a pristine root which I can then reapply to each PR. I can the submit 3 successive PR on that root: Proxy improvements, Global removal (and corresponding file reorg), and Reverse bridge. That will be the alpha/beta release that we can (hopefully) get some beta testers before we commit the shift. Honestly they will be large PR simply because global removal touches EVERYTHING and reverse bridge is nearly 50% more than our current code base.

I have the J2NI review in progress but they are being slow so it will likely be several months before we can show that prototype version. My hope with that is just getting it benchmarked and converted first. Because it is transaction based and supports bulk transfer it will be better for database support. The potential is there for improvements with JPype, but we have a lot of strange mechanics (as I was once again reminded) so actual mileage may vary.

[Thrameos]

I finished the global removal with a week to spare. There key realization was that to get rid of globals required making only a few adjustments:

• The _new or _init function must figure out how to get to the module state
• It then must cache it into the base type so that every member has it available as O(1)
• We then need to push it through every level with the frame we already were pushing around.
• Then Java needs to remember to pass the context back to every place that needs it

That then just left all the technical debt where we had failed to pass the frame through and places were the patterns need to shift to support subinterpreters.

[pelson]

I haven't got particularly strong opinions on any, but my order of preference (most important first):

1. unified gc - if this is letting the GC know accurately about objects that live on the other runtime, then this is pretty important for my user's usage, where data is streaming in frequently in Java, and then being handled in Python. The original issue/demonstrator that I raised was BUG report: JProxy instance passed to JClass constructor not destroyed when out of scope #581.
2. memory model - if this is addressing on exit issues (and seg faults) - Non-deterministic SIGSEGV with pytest + log4j #934
3. (not on the list) JForwarding, to use idiomatic imports without breaking the runtime behaviour expectation of Python (and definitely no running JVM) - Discussion: Minimising code execution at import time / the downsides of the JPype import system #933
4. reverse bridge - there are definitely cases where we would use JPype to run Python from Java. Today we do this via gRPC and out-of-process, but it is definitely appealing to be able to run subinterpreters from Python.

[Thrameos]

@pelson did the revised prototype for forward using with seem useful? Our problem with forwarding is we need to declare types or have an expensive side process read jars and identify symbols and cache it for future lookups.

My priority order right now is reverse bridge (usability) and j2ni (speed) as both if those are free with requiring cpython upstream changes.

[Thrameos]

@marscher I have a 99% functional copy of the reverse bridge running. It is just chasing down those last downcalls in the unit test. If you can tell me how to arrange the PRs then I can start assembling.

[marscher]

@Thrameos That's exciting news! I think we should prioritize a low integration effort first. When I remember correctly you said earlier, that j2ni needs a modern Java version, right?
So maybe we would have the bridge first. What do you say?

[Thrameos]

I was more looking for how to stage this into the repo. As one of those "Karl had to move many pieces" PR it has 20k lines of differences which is very punishing on you. I tried to stage it into 3 sets with core, global removal, than reverse bridge, but of course best laid plans as I found issues when assembling the pieces that required rework meaning it got all tangled up. (I had to flatten 4 times already)

So we just released 1.7.x series and if we want to only do core changes in can pull out a clean globals/core fix from the reverse bridge. That would let us make a development branch for reverse. As a huge API change that we are going to support long term we will need some serious review.

With the reverse bridge, most issues I have faced were in practice overloading issues. That is I put a face for foo(PyObject) and foo(Iterable<>) as both work on the back end but then I pass in a PyList and it triggers ambiguous overloads. I am then forced to rename to fooFromIterable(Iterable<>) or decide the user needs to cast there. In some cases the ended up just as foo(Object) because the behaviors are too wide. What is going on underneath is every dispatch form tuple(x) vs tuple(*x) must be given a unique name in the Java side. Just having the test bench isn't enough because with a dynamic casting system there are random this interface inherits from x,y,z this from x,z and so forth.

Given the complexity i still recommend the original ordering. I should branch the repo to a development copy with a linter, flatten to core only PR1, global on core PR2, then reverse bridge PR3. That would make the review process somewhat tolerable for you and means we can decide to pull in PR2 as 1.8.x. But once I run the linter it will break all outstanding PR, hence we need to decide if anything else is salvagable and close the rest or they will all end up as orphans.

J2NI is about 4 times more complex. Because to replaces JNI, it guts the whole common area with type manager, JPContext, and JPJavaFrame taking the brunt of the damage (>80%), but the type faces also don't leave unscathed (>20%) . The key change is without JNI binding to do the work, you instead must push C proxies into Java. It is actually a helpful change as rather than 20 random entry points you get a single point.

I hit first light last week on J2NI stand alone internal testing, discovered that to support multiple connections I needed a rework, completely it, then started working on networking, but discovered I need to rework my rework to support versioned ABI. I don't think the JPype given it's type system will be able to run multiple remotes at a time, but the reverse direction with Java spinning many remote copies of Python (each running JPype) appears doable. But even best case I don't see a working JPype2 prototype appearing until December.

I am also not quite sure what people will do with it. Over socket will run about 40% slower and lose some features, but it gain complete isolation. Java benefits much more than Python as you can spin up Python instance then close the connection and all the resources flush. Then spin another. So it's value is mostly depending on people actually using the reverse bridge which is as yet unproven.

My side quest this weekend was adding in inline encryption so that the connection is not a free remote execution exploit for any MiTM. But I am sure any cryptographic expert would find by mutating crypto state horrifying as I built the most fragile encryption protocol ever where a single bit out of place causes the remote connection to unravel and go back to reauthirization/reconnect.

[Thrameos]

Status report on this task. I have switched back to the J2Ni task for two weeks as that is a larger project and I will lose momentum if I don't keep it in active view. I have complete round test Java to Java feeding back to locally. But the rubber hits the road when I try to run over network because that stress tests if I made a bad pattern in the API. Unfortunately, I found multiple that need to be resolved as well as requirements for the encryption to avoid remote exploits. The initial software release is still stalled, so I need find a why to kick it.

I made decent progress and will try to press for a java to java execution over remote in 3 weeks. Once that works I will have enough design parameters to finish the C++ side. That would finally be enough to plan out the changes need to jpype native side.

Meanwhile I wrapped Llama.cpp in panama wrappers for direct orchestration without any middleman.

[Thrameos]

@ryanmkurtz Regarding the J2NI, are you likely to be able to use it? Under J2NI we should be able to support.

1. Python launched to one local JVM
2. Python launched to one remote JVM.
3. Java launched to local Python (with subinterpreters, we get this with reverse bridge)
4. Java launched to multiple remote Python. (including docker mounted images)

The only configuration we can't support easily is mixed (remote and local) or Python launched multiple connections (unless somehow we can rearrange our API to have a "jvm" concept for limited access).

Network conductivity will be Unix pipes or TCP sockets. The establishment connection will be TLS/SSL and the execution/data bus will be TCP with encrypted state machine. I will be working locally to get a security review obviously as that is a lot of surface area for remote exploits.

[Thrameos]

@pelson Given you are currently using gRPC I think that J2NI will also fit your use case. Under J2NI Java or Python can talk over a shared multiplexer bus line with up to 64 threads. So not quite the same as a full local process, but should cover most use cases.

[ryanmkurtz]

Sorry what exactly is J2NI?

[Thrameos]

I am replacing the foundation of JPype from JNI to Panama. That means creating a new JVM substrate protocol similar to JDWP and CapTP. It allows for in process message sharing and over wire using the same execution model. That allows the same JPype API to run both forward and reverse over many types of connections. Thus you can run in process (direct memory sharing) or remote (thread and memory mirroring) natively.

Create an isolated JVM

jpype.startJVM(spawn=True)

Create an isolated JVM with a fixed port

jpype.startJVM(spawn=True, remote="127.0.0.1:6698")

Link to an existing JVM

jpype.startJVM( remote="ssl://remote:9912")

Possibly one day?

with jpype.JVM(spawn=True) as jvm:
    hello = jvm.JClass("org.something.Hello")()
    hello.call(jvm.JString@"yes", 1,2)

Is that clear? Would it be useful?

[ryanmkurtz]

I think that is clear. So will there always be separate processes for the JVM and Python, as opposed to how the JVM currently resides in the Python process? Will there be a mode where Python can live within the JVM processes?

[Thrameos]

Yes on both counts. You get to select local execution (shared buffers to numpy work) or remote execution (all features except shared buffers work). You may select only one option though as having both a local and remote means that the tables that support will get trampled. If you are running locally you get 1) Python starts JVM (no Python subinterpreters, with buffer support, no isolation just like current JPype) OR 2) JVM starts Python ( Python subinterpreters and buffer support, no isolation) If you are running with remote then it may spawn a copy or use a specified copy. You get either 1) Python starts JVM (no Python subinterpreters, no buffer support, isolation) OR 2) Java starts JVM (multiple remote Python interpreters, no Python subinterpreters, no buffer support, isolation) Either way Java starts will support multiple interpreters, but there is a big difference. With remote interpreters you can use any module including those that do no support subinterpreters. With local you get a Main interpreter (any module) and then subinterpreters (only special modules with subinterpreter support.) Python starts are more limited as they are mainly to run one JVM.

[ryanmkurtz]

I think that would be useful indeed. I think Ghidra's first use of it would be to get its Java-driven Python interpreter running from the main Java process, rather than needing to start up the Ghidra JVM in a new Python process. It's not ideal for the user since they have to decide ahead of time if they want to use the python interpreter, so they can start Ghidra with the special launch scripts that support it. Python is not normally a runtime requirement of Ghidra.

Out of curiousity, have you tried Ghidra/PyGhidra?

[Thrameos]

@ryanmkurtz Sorry I haven't had time to test PyGhidra yet. Though it is on my list.

[Thrameos]

I have first light on the J2NI C++ interface testing. The results show J2NI's design philosophy pays off:

Bulk operations (the real use case):

• Marshalling arrays of objects: 34x faster than JNI
• Unmarshalling collections: 19x faster than JNI
• Array creation: 5x faster than JNI
• Overall average: 3.78x faster

Individual operations:
Near parity on lightweight ops (method calls, field access) with FFM boundary overhead, but that's fine - real applications spend their time marshalling database rows and bulk data, not making individual field
accesses.

The architectural wins (not measured in benchmarks):

• Zero GC pinning (no pauses during data transfer)
• Loom-friendly (virtual threads work safely)
• Zero-copy array mapping
• Modern, maintainable Java implementation

J2NI vs JNI Performance Benchmark

┌───────────────────────────┬───────────┬──────────┬─────────┬──────────┐
│ Benchmark │ J2NI (ms) │ JNI (ms) │ Speedup │ Winner │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ Bulk Marshal (100 ints) │ 0.55 │ 18.77 │ 34.25x │ J2NI ✓✓✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ Bulk Unmarshal (100 ints) │ 0.72 │ 13.83 │ 19.08x │ J2NI ✓✓✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ Array Creation │ 7.86 │ 38.91 │ 4.95x │ J2NI ✓✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ Integer Boxing │ 10.03 │ 16.26 │ 1.62x │ J2NI ✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ Integer Unboxing │ 8.22 │ 9.98 │ 1.21x │ J2NI ✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ Global Ref Churn │ 10.90 │ 8.78 │ 0.82x │ JNI ✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ Weak Ref Churn │ 11.80 │ 9.62 │ 0.82x │ JNI ✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ Array Modification │ 2.58 │ 2.02 │ 0.78x │ JNI ✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ Method Call (No Args) │ 14.39 │ 9.38 │ 0.65x │ JNI ✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ Method Call (With Args) │ 15.50 │ 10.01 │ 0.65x │ JNI ✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ Array Sum (Copy) │ 3.23 │ 2.06 │ 0.64x │ JNI ✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ String Creation │ 35.80 │ 18.98 │ 0.53x │ JNI ✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ String Length │ 19.66 │ 10.07 │ 0.51x │ JNI ✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ Array Sum (Critical) │ 3.77 │ 1.86 │ 0.49x │ JNI ✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ Local Ref Pressure │ 15.62 │ 6.73 │ 0.43x │ JNI ✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ Frame Push/Pop │ 7.48 │ 2.67 │ 0.36x │ JNI ✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ Array Slice Processing │ 10.31 │ 2.16 │ 0.21x │ JNI ✓ │
├───────────────────────────┼───────────┼──────────┼─────────┼──────────┤
│ Multiple Field Access │ 10.02 │ 1.00 │ 0.10x │ JNI ✓ │
└───────────────────────────┴───────────┴──────────┴─────────┴──────────┘

[marscher]

That's exiting news and very impressive benchmark results! I'm looking forward to see this in action.

[Thrameos]

I am guessing 80 to 160 hrs remain before I will have jpype2 prototype.

[Thrameos]

Another long weekend on future projects. This time on my webfx now renamed due to conflicts. Should be hitting the net in next two weeks so I had to spend the time catching up on release tasks for the office. Hopefully it will be useful to others.

[Thrameos]

I have almost gotten through the list of all the TODOs.

Highlights:

• Subinterpreter is now fully working which means you can spawn multiple interpreters from java each running JPype in full and cleaning up resources as they go.
• JPype is now a true multipart start.
• We are partially migrated to J2NI Global pooling structure. That was a requirement for automatic clean up of resources abandoned by closing subinterpreters. (but leak checking this ship will be a long process)
• SPI is complete so 3rd parties can create wrappers for NumPy, Pandas, and Torch if they so desire.
• SPI was tested by making a wrapper for python.io library, we now have full communications.
• Dozens of edge cases were dealt with including what happens if you call resources from one interpreter to another.
• Inception is possible in which you stack subinterpreters inside subinterpreters using the SubInterpreterBuilder.
• Unittesting is largely complete.
• Competitor analysis is complete. I have analyzed what features each package provides, and where gaps are found for JEP, JPY, Pyjnius, and Chaquopy. I then designed a corresponding API which was then implemented.
• Many bugs were stomped; much tea as consumed; several sunrises were seen from the wrong end.

Key issues going forward:

• Packaging needs to change to make it a true dual pom.xml/pip package so src/pom.xml will move to the root.
• Code formatting is ready to be applied.
• I will make the "alpha" branch for users to explore, but before that I would like resolve all issues and rebase everything so alpha is a clean shot from our existing repo.

@marscher Again sorry for the massive stream of PRs. I have been working to review them before they get to you, but they are coming in fast because I set up an automatic system using agents to analyze and propose solutions.

[Thrameos]

I completed the benchmarking between JPype, JPY, and JEP. (which matches my previous checks against Pyjnius from a few years back.)

The basic conclusion for general method/field dispatch we are heavier. We have much richer objects and thus pay a far higher price. In some cases it is as little as 20%, often in the x2 range, while others like the int path where CPython lacks a "create int with meta class" are x5 worse. That is not unexpected as full features has a price. The int torture test we did against Graalvm is our absolutely worst possible path in the whole code. So if looping integers one at a time is you measure of success, sadly we lose hands down.

Where the story is more interesting is bulk transfers which is what those other codes generally claim to be good at, JPype wins by substantial margins and often shockingly so. We only lose on small arrays where our method overhead is important. Once the array size grows large or the dimensions get deep, JPype is in a class of its own. We scale! Upload rates to Java up to 28 times faster. Download rates between 1.6 and 3 times faster. JPype remains the Scientific tool of choice for moving between numpy and Java.

[Thrameos]

J2NI cleared review so I can progress with demonstration and benchmarking of it. Estimate time 4 weeks to completion.