Bundle Python, create virtual environment and install datasette on startup

[simonw]

1. Bundle a copy of standalone python inside the application folder itself
2. On the first start up, create a virtual environment in ~/.datasette.app/venv using python -m venv with that bundles Python
3. Run bin/pip install datasette in that virtual environment

Originally posted by @simonw in #22 (comment)

[simonw]

I built a prototype of this using https://github.com/indygreg/python-build-standalone/releases/tag/20210724 and it worked surprisingly well!

The key challenge I am solving here relates to plugins, see #5. In order to install plugins I need to have a full Python environment with the entire standard library, in case any future plugins need to use that.

I need to be able to run pip install successfully. I need a Python installation that I can trust has everything I need, and that is completely independent from anything else the user may have already installed on their machine.

So... I'm going to try bundling a full standalone Python from https://github.com/indygreg/python-build-standalone in the Datasette.app application that I distribute. On startup, this will be used to create a virtual environment in ~/.datasette-app/venv which will then be used to install both Datasette and the datasette-app-support plugin.

This will then be launched by Electron.

Is this a good idea? I honestly don't know, but I'm going to get a working version of it up and see what people think.

[simonw]

https://pypi.org/project/datasette-app-support/ is now on PyPI.

[simonw]

In development mode this expects there to be a python folder inside datasette-app with the standalone Python bundle unzipped into it.

This works:

wget https://github.com/indygreg/python-build-standalone/releases/download/20210724/cpython-3.9.6-x86_64-apple-darwin-install_only-20210724T1424.tar.gz
tar -xzvf cpython-3.9.6-x86_64-apple-darwin-install_only-20210724T1424.tar.gz                                                                          
# Now delete the test/ folder, saving about 22MB of disk space
rm -rf python/lib/python3.9/test

[simonw]

Switching to a PR.

[simonw]

It totally worked!

I downloaded the artifact from https://github.com/simonw/datasette-app/actions/runs/1188102495 and installed the application from the DMG (to /Applications/Datasette.app) - then I ran it and it told me I couldn't because it wasn't signed.

So I went to Security & Privacy in the system preferences and clicked "open it anyway"... and it launched and installed everything it needed into ~/.datasette-app/venv and the application worked!