Get Datasette working with Zeit Now v2's 100MB image size limit

[simonw]

Follow-on from #366

Zeit Now's v2 cloud has a 100MB size limit on Docker images, in order to support much faster wake-ups of new instances.

Fitting Datasette AND the SQLite database it is hosting in here is going to be a challenge.

[simonw]

I think there is a useful way forward here though: the image size may be limited to 100MB, but once the instance launches it gets access to a filesystem with a lot more space than that (possibly as much as 15GB given my initial poking around).

So... one potential solution here is to teach Datasette to launch from a smaller image and then download a larger SQLite file from a known URL as part of its initial startup.

Combined with the ability to get Now to always run at least one copy of an instance this could allow Datasette to host much larger SQLite databases on that platform while playing nicely with the Zeit v2 platform.

See also vercel/vercel#1523

[simonw]

It turned out Zeit didn't end up shipping the new 100MB-limit Docker-based Zeit 2.0 after all - they ended up going in a completely different direction, towards lambdas instead (which don't really fit the Datasette model): https://zeit.co/blog/now-2

But... as far as I can tell they have introduced the 100MB image size for all free Zeit accounts ever against their 1.0 platform. So we still need to solve this, or free Zeit users won't be able to use datasette publish now even while 1.0 is still available.

I made some notes on this here: https://simonwillison.net/2018/Nov/19/smaller-python-docker-images/

I've got it working for the Datasette Publish webapp, but I still need to fix datasette publish now to create much smaller patterns.

I know how to do this for regular datasette, but I haven't yet figured out an Alpine Linux pattern for spatialite extras:

datasette/datasette/utils.py

Lines 287 to 300 in 5e3a432

	return '''
	FROM python:3.6
	COPY . /app
	WORKDIR /app
	{spatialite_extras}
	RUN pip install -U {install_from}
	RUN datasette inspect {files} --inspect-file inspect-data.json
	EXPOSE 8001
	CMD [{cmd}]'''.format(
	files=' '.join(files),
	cmd=', '.join(cmd),
	install_from=' '.join(install),
	spatialite_extras=SPATIALITE_DOCKERFILE_EXTRAS if spatialite else '',
	).strip()

[simonw]

This looks like it might be a recipe for spatialite Python on Alpine Linux: https://github.com/bentrm/geopython/blob/8e52062d9545f4b7c1f04a3516354a5a9155e31f/Dockerfile

[simonw]

Another example that might be useful: https://github.com/poc-flask/alpine/blob/8e9f48a2351e106347dab36d08cf21dee865993e/Dockerfile

[simonw]

Closing this as Zeit went on a different direction with Now v2, so the 100MB limit is no longer a concern.