consider using a separate python package to manage Julia dependency

[jlapeyre]

Here is a quick example:
https://github.com/jlapeyre/diffeq_julia

This handles installing Julia and Julia packages. It allows optionally compiling a system image particular to the diffeq project.

[ChrisRackauckas]

Yes, those are major issues I've been hoping to solve!!! Automated installation and #56. I would absolutely love to see a PR on this. I think it will greatly improve the library. Would you be willing to do the PR?

[jlapeyre]

I can probably find time over the next few weeks to make a PR that's a little more polished than the proof of concept that I posted above. But, I'd probably need help on the fine points of python packaging, e.g. setup.cfg, making sure CI works, etc.

[ChrisRackauckas]

Yeah, no worries. I will find the time, and hopefully @tkf could help.

[sibyjackgrove]

I just tested this and found this issue. jlapeyre/diffeq_julia#1
Just thought I'll mention it in case someone else is finding similar issues.

[jlapeyre]

I just tested this and found this issue. jlapeyre/diffeq_julia#1

A couple of problems turned up in that issue, both because PyCall can't be used with more than one libpython. It's a major thorn that has received a lot of attention.

I made some changes to julia_project, which is the python package that manages a Julia dependency in a python package. It detects when this problem arises upon importing the python module, eg import diffeq_julia. It then offers to perform one of the two most common solutions. It also offers some tools in case you know of the problem in advance.

The main goal of julia_project is to manage the Julia dependency without the Python end user learning or knowing anything at all about Julia.

[jlapeyre]

@sibyjackgrove helped to debug https://github.com/jlapeyre/diffeq_julia .

The package depends on https://github.com/jlapeyre/julia_project, which factors out managing a Julia project within a python package.

The latter depends in turn on https://github.com/jlapeyre/find_julia, which handles finding a path to a Julia exectuable, and if not found installing Julia via jill.py.

The directory with the system-image-compilation scripts is here:
https://github.com/jlapeyre/diffeq_julia/tree/main/diffeq_julia/sys_image

compile_exercise_script.jl is currently a dummy example. If it were replaced by something to exercise the Julia differential equations packages, then latency would be reduced. However, as it stands, compilation reduces startup time significantly.

EDIT: I think this should perhaps get a bit more testing in the wild before a PR is made to diffeqpy. The tests in julia_project and find_julia are minimal at best. Testing properly would require a lot of mocking of the state of your system. i.e. if and where julia is installed. If PyCall is installed, how it is built, etc. All the debugging was around these issues.

[jlapeyre]

I added a compile_exercise_script.jl to the code that builds the system image. It runs a few examples that should be compiled into the system image. This seems to have at best a small effect on the latency of running the python examples in the diffeqpy README.

[ChrisRackauckas]

This is now handled by PythonCall.