Test Datasette Docker images built for different architectures

[simonw]

Continuing on from #1319 - now that we have the ability to build Datasette's Docker image against multiple architectures we should test that it works.

We can do this with QEMU emulation, see https://twitter.com/nevali/status/1397958044571602945

[simonw]

I'm going to release a 0.57a1 alpha to Docker Hub using the new buildx mechanism.

[simonw]

(Should also update https://docs.datasette.io/en/stable/contributing.html#release-process with notes on how this works)

[simonw]

https://github.com/simonw/datasette/runs/2687196350 is a run of https://github.com/simonw/datasette/blob/main/.github/workflows/push_docker_tag.yml to publish the new 0.57a1 alpha to Docker Hub.

[simonw]

Related issue: #1272 - being able to run the test suite inside Docker inside GitHub Actions would be incredibly helpful here.

[simonw]

Job completed in 13m38s - results are here: https://hub.docker.com/r/datasetteproject/datasette/tags?page=1&ordering=last_updated&name=0.57a1

[simonw]

Having pushed that to Docker Hub the following seems to work on my laptop:

docker run --platform linux/s390x \
  -v `pwd`:/mnt -p 8001:8001 \
  datasetteproject/datasette:0.57a1 datasette -p 8001 -h 0.0.0.0

I'd like to confirm that this is correctly emulating the architecture and running the correct published image though.

[simonw]

This helps:

~ % docker run -it --platform linux/s390x -v `pwd`:/mnt \
  -p 8001:8001 datasetteproject/datasette:0.57a1 \
  python -c 'import platform; print(platform.uname())'

uname_result(system='Linux', node='d14916ca91df', release='4.19.121-linuxkit', version='#1 SMP Thu Jan 21 15:36:34 UTC 2021', machine='s390x')

[simonw]

~ % docker run -it --platform linux/arm/v7 -v `pwd`:/mnt -p 8001:8001 datasetteproject/datasette:0.57a1 python -c 'import platform; print(platform.uname())'
Unable to find image 'datasetteproject/datasette:0.57a1' locally
0.57a1: Pulling from datasetteproject/datasette
ee0a2cc24f29: Pull complete 
ca231153300d: Pull complete 
6073fc53d406: Pull complete 
bb2a6a6421dd: Pull complete 
14c15f441034: Pull complete 
28d113f72ba5: Pull complete 
40516c2c3785: Pull complete 
Digest: sha256:8dd469efa0b34b9e946c4e91496fc7bec702efec9c0f5ec548ccd4030f3263b2
Status: Downloaded newer image for datasetteproject/datasette:0.57a1
uname_result(system='Linux', node='c2e1a05732ed', release='4.19.121-linuxkit', version='#1 SMP Thu Jan 21 15:36:34 UTC 2021', machine='armv7l')

Shorter version:

~ % docker run -it --platform linux/ppc64le datasetteproject/datasette:0.57a1 python -c 'import platform; print(platform.uname())'
Unable to find image 'datasetteproject/datasette:0.57a1' locally
0.57a1: Pulling from datasetteproject/datasette
c840eb5e9aed: Pull complete 
fc17d8b577ec: Pull complete 
712a0ad995f8: Pull complete 
56101149914c: Pull complete 
e6241dfe9d38: Pull complete 
41ed85039888: Pull complete 
26beeabe9213: Pull complete 
Digest: sha256:8dd469efa0b34b9e946c4e91496fc7bec702efec9c0f5ec548ccd4030f3263b2
Status: Downloaded newer image for datasetteproject/datasette:0.57a1
uname_result(system='Linux', node='0fe85887fa30', release='4.19.121-linuxkit', version='#1 SMP Thu Jan 21 15:36:34 UTC 2021', machine='ppc64le')
~ % 

OK! Now I need to figure out how to run the full test suite on each one, then I can call this done.

[simonw]

To run the test suite... need to install a few more dependencies:

datasette/setup.py

Lines 71 to 79 in 7b106e1

	"test": [
	"pytest>=5.2.2,<6.3.0",
	"pytest-xdist>=2.2.1,<2.3",
	"pytest-asyncio>=0.10,<0.16",
	"beautifulsoup4>=4.8.1,<4.10.0",
	"black==21.5b1",
	"pytest-timeout>=1.4.2,<1.5",
	"trustme>=0.7,<0.8",
	],

And then mount the tests/ folder into the container and run pytest.

[simonw]

This pattern appears to work, executed at the root of a checkout of datasette:

docker run -it -v `pwd`:/mnt --platform linux/amd64 \
  datasetteproject/datasette:0.57a1 bash -c '
    pip install "pytest>=5.2.2,<6.3.0" \
      "pytest-xdist>=2.2.1,<2.3" \
      "pytest-asyncio>=0.10,<0.16" \
      "beautifulsoup4>=4.8.1,<4.10.0" \
      "black==21.5b1" \
      "pytest-timeout>=1.4.2,<1.5" \
      "trustme>=0.7,<0.8" \
    && cd /mnt && pytest'