epic: Jan has Python Runtime

[dan-homebrew]

Problems

• Extension developers would expend a great deal of effort to make their Python program function within an extension, as
  • To run a Python program, a user needs to have the proper Python version and required dependencies installed on their computer.
  • There is no out-of-the-box communication channel between the Python program and the app's processes (both main and renderer processes).

For example, when converting HuggingFace models to the GGUF format, the extension starts a Python process to install all necessary Python dependencies on the user's computer. Regardless of the user's installed Python or pip version, they wait for the installation to complete, then start another Python process to convert the model and wait for it to finish.
Here are the corrections:

• No version checking for Python, nor an onboarding process.
• No state synchronization between processes, waits only for exit codes 0 or 1.
• Inefficient process: chaining the installation of dependencies every time a Python program is run.
• Poor user experience for error handling.
• It is a pass/fail program, with no data emitted.
• We install dependencies, without sandboxing, into the user's site-packages directory. It is difficult to clean data while uninstalling the application.
• Must manually bridge over main process.

Objective

• Easy to build: extension builders should not care about how to spawn a Python program using Node's child_process or its dependencies, nor about the Python version.
• Sandboxed: everything bundled with the extension; no extra installation required.
• Seamless: A simple communication protocol across multiple processes and languages.

Context

1. Extensible python host ❌

Electron main process & python host lifecycle

graph LR
    A[Electron Main Process] -- runs watchdog --> E[Watchdog Process]
    E -- monitors --> B[Python Server]

Loading

Execution

graph TD
    A[Electron Main Process] --> B[Python Server]
    B --> C[Extension]
    C --> D[Application]
    classDef runtime fill:#f9d2d2,stroke:#333;
    class C runtime;

Loading

Problems:

• Supports only one Python version, very limited. Dependency management is challenging, and it is not sandboxed.
• Bloating the main Electron app makes it hard to manage the extension lifecycle.

2. Extension manages it's python processes
   a. Python and its dependencies are bundled using PyInstaller, making distribution across platforms easy. 👍
   - An extension that supports multiple Python versions would distribute different executable files.
   - The executable file must be signed.
   - Some of the dependencies may not have been bundled properly.

   b. Virtualenv ❌
   - Hard to find the exact python endpoint.
   - They're not really intended to be copied around.
   - They have a bunch of symlinks and absolute paths in them (plus os-dependent stuff).
   - Difficult to perform code signing (due to the large number of files and nested levels).

SDK Update

1. Communication protocol 👍

   • Using zeromq or zerorpc... for a better communication across processes.

     graph LR
       A[Main Process] <--streaming-rpc--> B[Renderer Process]
       A <--streaming-rpc--> C[Extensions]
       A <--streaming-rpc--> D[Python Processes]
     

     Loading
   • Communicate using core SDK instead of adding a dependency and setting up a server/client.
   • Decorators for a clean extension codebase.
   • Import JAN Core SDK directly into JS/TS extensions and Python scripts.

2. Bundler 👍

   • Package.json should have a pre-defined step to compile the python source directory into an executable file.
   • Index.ts still acts as the main entry point, setting up the UI and configurations, and serving as a middleware for frontend components and the Python program.
   • Running and communicating with a Python program should be straightforward.

[0xSage]

Dupe of #555