Package datasette for installation using homebrew

[simonw]

https://docs.brew.sh/Python-for-Formula-Authors describes how.

Applications should be installed into a Python virtualenv environment rooted in libexec. This prevents the app’s Python modules from contaminating the system site-packages and vice versa.

It recommends using https://github.com/tdsmith/homebrew-pypi-poet

[simonw]

Here are some useful examples of other Python apps that have been packaged using the recipe described above: https://github.com/Homebrew/homebrew-core/search?utf8=%E2%9C%93&q=virtualenv_install_with_resources&type=

[simonw]

Huh... from https://docs.brew.sh/Acceptable-Formulae

We frown on authors submitting their own work unless it is very popular.

Marking this one as "help wanted" :)

[simonw]

I can host a custom tap without needing to get anything accepted into homebrew-core: https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap

Since my principle goal here is ensuring an easy installation path for people who are familiar with brew but don't know how to use pip and Python 3 that could be a good option.

[simonw]

Here's a good example of a homebrew tap: https://github.com/saulpw/homebrew-vd

[simonw]

Another useful example: https://github.com/Homebrew/homebrew-core/blob/master/Formula/trailscraper.rb

[simonw]

How would plugin installation work if Datasette was installed via homebrew?

[simonw]

Here's what happened when I installed homebrew-vd: https://gist.github.com/simonw/7bfd971a62743d7ca248e6b5e696c240

It worked! And from digging around, it has a virtual environment at /usr/local/Cellar/visidata/1.5.2/libexec/

Which means /usr/local/Cellar/visidata/1.5.2/libexec/bin/pip is a working pip

And I tried running these commands and confirmed that I get a datasette with an additional plugin:

/usr/local/Cellar/visidata/1.5.2/libexec/bin/pip install datasette
/usr/local/Cellar/visidata/1.5.2/libexec/bin/pip install datasette-graphql
/usr/local/Cellar/visidata/1.5.2/libexec/bin/datasette plugins
[
    {
        "name": "datasette-graphql",
        "static": false,
        "templates": true,
        "version": "0.11",
        "hooks": [
            "register_routes",
            "startup"
        ]
    }
]

So I can package Datasette as a homebrew package AND I can give people instructions for installing plugins.

[simonw]

Telling people how to figure out that pip location is going to be pretty unpleasant.

How about instead providing a datasette plugins --install=datasette-graphql command?

Or datasette install datasette-vega

It would run pip install in the same virtualenv as Datasette itself.

http://jelly.codes/articles/python-pip-module/ shows how to do this:

import pip

try:
    pip.main(["install", "plumbum"])
except SystemExit as e:
    pass

[simonw]

Quick prototype of datasette install:

diff --git a/datasette/cli.py b/datasette/cli.py
index 287195a..95b6eb7 100644
--- a/datasette/cli.py
+++ b/datasette/cli.py
@@ -231,6 +231,18 @@ def package(
         call(args)
 
 
+@cli.command()
+@click.argument("packages", nargs=-1, required=True)
+def install(packages):
+    "Install Python packages - e.g. Datasette plugins - into the same environment as Datasett"
+    import pip
+
+    try:
+        pip.main(["install"] + list(packages))
+    except SystemExit as e:
+        pass
+
+
 @cli.command()
 @click.argument("files", type=click.Path(exists=True), nargs=-1)
 @click.option(

$ datasette install
Usage: datasette install [OPTIONS] PACKAGES...
Try 'datasette install --help' for help.

Error: Missing argument 'PACKAGES...'.
$ datasette install datasette-vega
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Collecting datasette-vega
  Using cached datasette_vega-0.6.2-py3-none-any.whl (1.8 MB)
...

[simonw]

Here's the issue that explains that warning: pypa/pip#5599

This should fix it (risky):

from pip._internal.cli.main import main

[simonw]

I got this almost working in simonw/homebrew-datasette - see simonw/homebrew-datasette#2 for the last missing detail.

[simonw]

It works!

$ brew tap simonw/datasette
$ brew install simonw/datasette/datasette
$ datasette --version
datasette, version 0.46