Example: Python backend for Nextron

[yoDon]

I've been working on a Python backend example based on https://github.com/fyears/electron-python-example. fyear's example uses zerorpc to enable electron renderer javascript to talk to a python backend.

As described in the README.md for the example, this PR uses conda (part of the Anaconda Python distribution, essentially npm for Python) to make sure all the necessary Python libraries are installed. Conda is super easy BUT for some reason it doesn't run correctly under Powershell on Windows. That means if you use VS Code on windows you need to turn your Powershell integrated terminal editor into a DOS command prompt by typing cmd in the editor before you use conda. The way conda manages environments also means if you close VS Code and reopen it, you need to cmd and conda activate ... again, as described in the README.md (I know that sounds complicated but it's actually surprisingly simple and all this is pretty well explained in the readme).

There is one slight wall I hit in putting this example together. There is a .node file (compiled binary node executable) that is part of the zerorpc networking layer the processes use to intercommunicate. fyear's example has no problem with this zerorpc communication between electron renderer and python. I was able to get the .node file loaded by the next server side rendering code (so it loads when the page is run through SSR) but I wasn't able to get it to load in the actual client side renderer under nextron. All of this is detailed in the final paragraph of the example's README.md I'm pretty sure this is a Nextron webpack issue, because fyear's sample code is unmodified electron and react (no webpack files) and everything runs great. I had to add the native-ext-loader to next.config.js to get the server side rendering to work for the pages/calculator page but that didn't seem to be enough to get things to work clientside. I do believe the client issue is a loader issue because if I uncomment any of the zerorpc code in pages/calculator in order to enable it, all processing of the pages/calculator page stops working silently (with none of the console.log() statements in the page printing anything to the client console, even at the front of the page before any zerorpc commands are executed).

It's soooo close and fyear's example is so simple... it'd be really slick to be able to use Python on the backend for machine learning or whatever, but I don't know enough webpack to be able to unblock this.

[yoDon]

From the README.md of the example:

There is a node binary which needs to be loaded for this sample to work (node_modules/zeromq/build/Release/zmq.node, which is required by zerorpc). fyear's original Electron+Python example didn't seem to need any special handling for this, but I found I needed to modify /renderer/next.config.js to add the native-ext-loader to webpack in order to get nextron to load the node_modules/zeromq/build/Release/zmq.node node binary executable. That change to next.config.js allows the server to load zeromq (which is used by zerorpc) but it doesn't seem to be sufficient to allow the client to load it. Any reference to zerorpc on the client causes the client code to fail silently (see the TODO DEBUG comments in pages/calculator/index.tsx), which is blocking use of this sample app.

[saltyshiomix]

Hi @yoDon !
Thank you for your PR! This is my first PR, I'm very glad to 🥇

[saltyshiomix]

I found python@3.x doesn't work this example.
(python@2.7 works 👍 )

So it would be greater if your README has requirements section :)

[saltyshiomix]

One more thing.

Nextron's examples folder is used by a nextron init command as templating.
So would you move examples/with-typescript-python to showcases/with-typescript-python like?

[saltyshiomix]

The idea of python backend is awesome :)

[yoDon]

Will do! If you have any thoughts on how to get the webpack (or ?) issue with the zerorpc client working, I'm pretty sure I could also get a .NET backend running very quickly as it would be almost identical to the Python showcase.

[saltyshiomix]

The command nextron.js build internally uses electron-builder and it packages app directory by default.
So in production build, you should include pythondist into app directory before running electron-builder.
And finally, you must resolve path to pythondist from main process according to build mode (process.env.NODE_ENV === 'production' or not)

[yoDon]

I've updated to Nextron 3.3.9, changed the Python <-> Electron communication from ZeroRPC to GraphQL because ZeroRPC's binary .node files are apparently hard to package in Electron, and updated the README.md including mentioning that I did all my testing on Python v3 while developing this.

There is only one substantive change I needed to make to the templated Nextron files: I added one line to background.ts to add the python launching code to the main app launching process. The other changes I made were all trivial/automated formatting changes to get the typescript code to pass the tslint linter without errors.

The GraphQL client is created in _app.tsx and used in calculator/index.tsx (this pattern allows for better caching of results in real application than creating the GraphQL client in an individual page like the calculator page).

There is one weak spot in this example that I suspect could be greatly improved using a technique similar to your new ipc resolver. In main/addPython.ts I was battling the same kind of is-this-running-in-a-packaged-electron-app question that you were battling when writing the ipc resolver. In my case it was to determine which syntax to use for launching the Python server.

I also just noticed that the addPython.ts script has some leftover port defining logic that was used by the old ZeroRPC communication channel but isn't wired into the new Flask/GraphQL communication logic. I probably should do that last bit of tweaking but it's basically working and I don't have time to do that change tonight so I'm going to go ahead and send this update now and hopefully tweak the port handling in a few days or next weekend.

[yoDon]

PR significantly cleaned up/improved and converted to template format.

From the new Readme.md:

Electron + Next.js + Python Flask + GraphQL

This example builds a stand-alone Electron + Next + Python application and installer. On Windows it builds the app into ./dist/win-uppacked/My Nextron Python App.exe and the installer into ./dist/My Nextron Python App Setup 1.0.0.exe (OSX and Linux destinations are similar). You can change the name of the application by changing the name property in package.json.

Installation

Tested with Anaconda Python v3, should work fine with Anaconda Python v2.

NOTE: On windows you will need to install anaconda (which installs python and pip) and potentially configure environment variables to add python and/or pip to the path if you don't have it installed already.

# start with the obvious step you always need to do with node projects
npm install

# Depending on the packages you install, with Electron projects you may need to do 
# an npm rebuild to rebuild any included binaries for the current OS. It's probably
# not needed here but I do it out of habit because its fast and the issues can be
# a pain to track down if they come up and you dont realize a rebuild is needed
npm rebuild

VERY IMPORTANT: Windows users, if you use VS Code or use Powershell as your shell, you need to type cmd inside the VS Code terminal or inside your Powershell window before running the conda commands because conda's environment switcher will not work under Powershell (much of it works, but the critical parts that don't work, like activating evironments, fail silently while appearing to work),

# install Anaconda if not already installed

cmd # Only needed if you're coding on Windows in VS Code or Powershell, as discussed above
conda env create -f environment.yml
conda activate nextron-python-sample
conda env list # make sure the nextron-python-sample has a * in front indicating it is activated (under Powershell on Windows the activate command fails silently which is why you needed to run the conda commands in a cmd prompt)

# run the unpackaged python scripts from a dev build of electron
npm run dev # must be run in the same shell you just conda activated

NOTE if you see the following error message when trying to npm run dev it means you did not successfully conda activate nextron-python-sample in the shell from which you are trying to npm run dev. On Windows under VS Code that could be because you forgot to go into a cmd shell as discussed above before trying to conda activate.

Traceback (most recent call last):
  File "python/api.py", line 3, in <module>
    from graphene import ObjectType, String, Schema
ModuleNotFoundError: No module named 'graphene'

# use pyinstaller to convert the source code in python/ into an executable in pythondist/, build the electron app, and run electron-packager to package the electron app as a single file
npm run build # must be run in the same shell you just conda activated

# double-click to run the either the platform-specific app that is built into a subdirectory of dist or the platform-specific installer that is built and placed in the dist folder

Debugging Python GraphQL server

To test the python GraphQL server, in a conda activated terminal window run npm run build-python, cd into the newly generated pythondist folder, and run api.exe then browse to http://127.0.0.1:5000/graphql/ to access a GraphiQL viewer of the GraphQL server. For a more detailed example, try http://127.0.0.1:5000/graphql/?query={calc(math:"1/2")} which works great if you copy and paste into the browser but which is a complex enough URL that it will confuse chrome if you try to click directly on it.

Notes

The electron main process both spawns the Python child process and creates the window. The electron renderer process communicates with the python backend via GraphQL web service calls.

The Python script python/calc.py provides a function: calc(text) that can take text like 1 + 1 and return the result like 2.0. The calc functionality is exposed as a GraphQL api by python/api.py.

The details of how the electron app launches the Python executable is tricky because of differences between packaged and unpackaged scenarios. This complexity is handled by main/background-with-python.ts. If the Electron app is not packaged, the code needs to spawn the Python source script. If the Electron app is packaged, it needs to execFile the packaged Python executable found in the app.asar. To decide whether the Electron app itself has been packaged for distribution or not, main/background-with-python.ts checks whether the __dirname looks like an asar folder or not. Killing spawned processes under Electron can also be tricky so the electron main process sends a message to the Python server telling it to exit when Electron is shutting down.

[saltyshiomix]

@yoDon

Thank you for your hard efforts 🥇
I'll test it tomorrow and if it's ok I'll merge it to the master and publish new version this weekend :)

[saltyshiomix]

@yoDon

Could you change require("child_process").spawn() to require('cross-spawn')() ?
Native spawn doesn't work on Windows. (Mac is fine, you done the things 👍 )

If you are busy, I'll merge this to the master and change to use cross-spawn before releasing new nextron :)

[yoDon]

Done! (I'm on the train with a laptop at ~0% battery so I wasn't able to test the change but it's pretty simple and hopefully typo-free)

[saltyshiomix]

@yoDon

The example with-typescript-python finally works on Windows 👍
Great works!!

[saltyshiomix]

@yoDon

Just released nextron@4.1.0 including your awesome PR 🎉

[saltyshiomix]

@yoDon

By the way, GitHub can't list you as a contributor because your local git email address is different from GitHub email address.

Could you add your local git email address as an alias of GitHub ones?
If you do that, GitHub can list you as a contributor:
(2 contributors => 3 contributors)