Documentation with recommendations on running Datasette in production without using Docker

[chrismp]

I've got some SQLite databases too big to push to Heroku or the other services with built-in support in datasette.

So instead I moved my datasette code and databases to a remote server on Kimsufi. In the folder containing the SQLite databases I run the following code.

nohup datasette serve -h 0.0.0.0 *.db --cors --port 8000 --metadata metadata.json > output.log 2>&1 &.

When I go to http://my-remote-server.com:8000, the site loads. But I know this is not a good long-term solution to running datasette on this server.

What is the "correct" way to have this site run, preferably on server port 80?

[simonw]

I'm still trying to figure this out myself.

I'm confident that running nginx on port 80 and using it to proxy traffic to Datasette is a sensible way to solve the port problem.

As for running Datasette itself: the two options that seem best to me are some kind of Init.d service or running it under supervisord. I have to admit I haven't worked out the necessary incantation for either of those yet: the solitary instance I have that's not running as a Docker container is sitting in a "screen" instance for the moment!

[simonw]

A section in the Datasette docs that acts as recommendations plus a tutorial for running Datasette on a VPS without using a Docker would be excellent.

[simonw]

This is also relevant to Datasette Library #417

[russss]

On most modern Linux distros, systemd is the easiest answer.

Example systemd unit file (save to /etc/systemd/system/datasette.service):

[Unit]
Description=Datasette
After=network.target

[Service]
Type=simple
User=<username>
WorkingDirectory=/path/to/data
ExecStart=/path/to/datasette serve -h 0.0.0.0 ./my.db
Restart=on-failure

[Install]
WantedBy=multi-user.target

Activate it with:

$ sudo systemctl daemon-reload
$ sudo systemctl enable datasette
$ sudo systemctl start datasette

Logs are best viewed using journalctl -u datasette -f.

[simonw]

I asked about this on Twitter and got a solid recommendation for systemd and this tutorial: https://www.digitalocean.com/community/tutorials/how-to-use-systemctl-to-manage-systemd-services-and-units - via https://twitter.com/sil/status/1142412145990221825

[simonw]

... and @russss also suggested systemd 21 seconds before I posted that!

[simonw]

Here are some partial notes I have saved from an nginx configuration I've used in the past:

cat /etc/nginx/sites-available/default
server {
	listen 80 default_server;
	listen [::]:80 default_server;

        location / {
                proxy_pass http://127.0.0.1:8001/;
		proxy_set_header Host $host;
        }
...

[simonw]

This example is useful to - I like how it has a Makefile that knows how to set up systemd: https://github.com/pikesley/Queube

[russss]

This example is useful to - I like how it has a Makefile that knows how to set up systemd: https://github.com/pikesley/Queube

I wasn't even aware it was possible to add a systemd service at an arbitrary path, but it seems a little messy to me.

Maybe worth noting that systemd does support per-user services which don't require root access. Cool but probably overkill for most people (especially when you're going to need root to listen on port 80 anyway, directly or via a reverse proxy).

[chrismp]

WorkingDirectory=/path/to/data

@russss, Which directory does this represent?

[russss]

WorkingDirectory=/path/to/data

@russss, Which directory does this represent?

It's the working directory (cwd) of the spawned process. In this case if you set it to the directory your data is in, you can use relative paths to the db (and metadata/templates/etc) in the ExecStart command.

[chrismp]

WorkingDirectory=/path/to/data

@russss, Which directory does this represent?

It's the working directory (cwd) of the spawned process. In this case if you set it to the directory your data is in, you can use relative paths to the db (and metadata/templates/etc) in the ExecStart command.

In my case, on a remote server, I set up a virtual environment in /home/chris/Env/datasette, and when I activated that environment I ran pip install datasette.

My datasette project is in /home/chris/datatsette-project, so I guess I'd use that directory in the WorkingDirectory parameter?

And the ExecStart parameter would be /home/chris/Env/datasette/lib/python3.7/site-packages/datasette serve -h 0.0.0.0 my.db I'm guessing?