Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" #1497

Closed
simonw opened this issue Oct 24, 2021 · 18 comments
Closed
Labels
bug ci docker The official Docker image, plus other things related to running Datasette on Docker

Comments

@simonw
Copy link
Owner

simonw commented Oct 24, 2021

This means the Datasette 0.59.1 release has not been published to Docker Hub.

Here's where that failed: https://github.com/simonw/datasette/runs/3991043374?check_suite_focus=true

Preparing to unpack .../libc6_2.32-4_amd64.deb ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.28.1 /usr/local/share/perl/5.28.1 /usr/lib/x86_64-linux-gnu/perl5/5.28 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.28 /usr/share/perl/5.28 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.)
debconf: falling back to frontend: Teletype
Checking for services that may need to be restarted...
Checking init scripts...
Unpacking libc6:amd64 (2.32-4) over (2.28-10) ...
Setting up libc6:amd64 (2.32-4) ...
/usr/bin/perl: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory
dpkg: error processing package libc6:amd64 (--configure):
 installed libc6:amd64 package post-installation script subprocess returned error exit status 127
Errors were encountered while processing:
 libc6:amd64
E: Sub-process /usr/bin/dpkg returned an error code (1)
The command '/bin/sh -c apt-get update &&     apt-get -y --no-install-recommends install software-properties-common &&     add-apt-repository "deb http://httpredir.debian.org/debian sid main" &&     apt-get update &&     apt-get -t sid install -y --no-install-recommends libsqlite3-mod-spatialite &&     apt-get remove -y software-properties-common &&     apt clean &&     rm -rf /var/lib/apt &&     rm -rf /var/lib/dpkg/info/*' returned a non-zero code: 100

Same problem when I attempted to publish using the "Push specific Docker tag" workflow: https://github.com/simonw/datasette/runs/3991059912?check_suite_focus=true

@simonw simonw added bug ci docker The official Docker image, plus other things related to running Datasette on Docker labels Oct 24, 2021
@simonw
Copy link
Owner Author

simonw commented Oct 24, 2021

I can replicate locally by running:

docker build -f Dockerfile \
          -t datasetteproject/datasette:0.59.1 \
          --build-arg VERSION=0.59.1 .

This gives me the same error.

@simonw
Copy link
Owner Author

simonw commented Oct 24, 2021

I got the same error publishing 0.59: https://github.com/simonw/datasette/actions/runs/1343251945

@simonw
Copy link
Owner Author

simonw commented Oct 24, 2021

Right now the base image is:

FROM python:3.9.2-slim-buster as build

I'm going to try python:3.9.7-slim-buster instead: https://hub.docker.com/layers/python/library/python/3.9.7-slim-buster/images/sha256-290b95e4b379762a9bd3d72644598e0972f4e2b5442bba60592c018fadcc744d?context=explore

@simonw
Copy link
Owner Author

simonw commented Oct 24, 2021

Same errors with 3.9.7:

#5 41.46 /usr/bin/perl: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory
#5 41.46 dpkg: error processing package libc6:amd64 (--configure):
#5 41.46  installed libc6:amd64 package post-installation script subprocess returned error exit status 127
#5 41.47 Errors were encountered while processing:
#5 41.47  libc6:amd64
#5 41.50 E: Sub-process /usr/bin/dpkg returned an error code (1)

I'm suspicious of this part of the Dockerfile:

datasette/Dockerfile

Lines 1 to 18 in e6e4437

FROM python:3.9.2-slim-buster as build
# Version of Datasette to install, e.g. 0.55
# docker build . -t datasette --build-arg VERSION=0.55
ARG VERSION
# software-properties-common provides add-apt-repository
# which we need in order to install a more recent release
# of libsqlite3-mod-spatialite from the sid distribution
RUN apt-get update && \
apt-get -y --no-install-recommends install software-properties-common && \
add-apt-repository "deb http://httpredir.debian.org/debian sid main" && \
apt-get update && \
apt-get -t sid install -y --no-install-recommends libsqlite3-mod-spatialite && \
apt-get remove -y software-properties-common && \
apt clean && \
rm -rf /var/lib/apt && \
rm -rf /var/lib/dpkg/info/*

@simonw
Copy link
Owner Author

simonw commented Oct 24, 2021

Looks like it's this bug, reported on the Debian mailing list: https://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg1818037.html

No obvious workaround there though.

@simonw
Copy link
Owner Author

simonw commented Oct 24, 2021

Here that is in the Debian bug tracker: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=993755

@simonw
Copy link
Owner Author

simonw commented Oct 24, 2021

I think the root cause here is that I'm using a Debian Buster base image and then installing SpatiaLite from Debian unstable (sid) - as described in this comment: #1249 (comment)

That's has worked fine in the past, but Sid is unstable - and this seems to be one of those instabilities.

@simonw
Copy link
Owner Author

simonw commented Oct 24, 2021

Debian stable these days is "bullseye" - https://www.debian.org/releases/ - which has the version of SpatiaLite that I was previously pulling in from Sid: https://packages.debian.org/bullseye/libsqlite3-mod-spatialite

So upgrading to the 3.9.7-slim-bullseye base image may help.

https://hub.docker.com/layers/python/library/python/3.9.7-slim-bullseye/images/sha256-67af5f544115124dc6d6da1d9d2815aa9825f6fd4aa6710adb0ec1725280fb89?context=explore

@simonw
Copy link
Owner Author

simonw commented Oct 24, 2021

That fixed it! Resulting image is 249MB which is a very slight size reduction (I think previous was 259MB (uncompressed).

@simonw
Copy link
Owner Author

simonw commented Oct 24, 2021

I'm going to attempt to publish 0.59 to Docker Hub using https://github.com/simonw/datasette/blob/2c31d1cd9cd3b63458ccbe391866499fa3f44978/.github/workflows/push_docker_tag.yml - if that works I'll push 0.59.1 as well.

@simonw
Copy link
Owner Author

simonw commented Oct 24, 2021

Testing that newly published image:

% docker run -p 8002:8001 -v `pwd`:/mnt \
    datasetteproject/datasette:0.59 datasette -p 8001 -h 0.0.0.0 /mnt/fixtures.db
Unable to find image 'datasetteproject/datasette:0.59' locally
0.59: Pulling from datasetteproject/datasette
7d63c13d9b9b: Already exists 
6ad2a11ca37b: Already exists 
e9edbe81a001: Already exists 
36629b83aba2: Already exists 
7338abefe51c: Already exists 
6d71b6b88b82: Pull complete 
8c4da3c56bdc: Pull complete 
Digest: sha256:038decc28e0ea84b281ecc0058fe8eba7aa99596e5a2177ff714092ad03294ed
Status: Downloaded newer image for datasetteproject/datasette:0.59
INFO:     Started server process [1]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit)

and http://localhost:8002/versions.json returns:

{
    "python": {
        "version": "3.9.7",
        "full": "3.9.7 (default, Oct 12 2021, 02:43:43) \n[GCC 10.2.1 20210110]"
    },
    "datasette": {
        "version": "0.59"
    },
    "asgi": "3.0",
    "uvicorn": "0.15.0",
    "sqlite": {
        "version": "3.34.1"

@simonw
Copy link
Owner Author

simonw commented Oct 24, 2021

One catch: the latest tag on Docker Hub is still three months old.

@simonw
Copy link
Owner Author

simonw commented Oct 24, 2021

That's because the publish.yml workflow ends with this, which isn't in the push_docker_tag.yml workflow:

docker tag $REPO:${GITHUB_REF#refs/tags/} $REPO:latest
docker push $REPO:${GITHUB_REF#refs/tags/}
docker push $REPO:latest

@simonw
Copy link
Owner Author

simonw commented Oct 24, 2021

Tried fixing this by pushing a new latest tag from my laptop:

(datasette) datasette % docker pull datasetteproject/datasette:0.59.1
0.59.1: Pulling from datasetteproject/datasette
7d63c13d9b9b: Already exists 
6ad2a11ca37b: Already exists 
e9edbe81a001: Already exists 
36629b83aba2: Already exists 
7338abefe51c: Already exists 
6b825daddc6c: Pull complete 
d7508b065a21: Pull complete 
Digest: sha256:dc134f65bec40ed4ea7049188fe1e3915b8e6c3fd999b17effe8ec24868b979c
Status: Downloaded newer image for datasetteproject/datasette:0.59.1
docker.io/datasetteproject/datasette:0.59.1
(datasette) datasette % docker tag datasetteproject/datasette:0.59.1 datasetteproject/datasette:latest
(datasette) datasette % docker push datasetteproject/datasette:latest
The push refers to repository [docker.io/datasetteproject/datasette]
d668c99b6ff1: Layer already exists 
aa20c9013575: Layer already exists 
c97eebf2b227: Layer already exists 
284a6c64b82c: Layer already exists 
388eedeb736e: Layer already exists 
2feece0964b8: Layer already exists 
e8b689711f21: Layer already exists 
errors:
denied: requested access to the resource is denied
unauthorized: authentication required

So I logged in with docker login:

(datasette) datasette % docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: datasetteproject
Password: 

And ran the push again and it worked:

(datasette) datasette % docker push datasetteproject/datasette:latest
The push refers to repository [docker.io/datasetteproject/datasette]
d668c99b6ff1: Layer already exists 
aa20c9013575: Layer already exists 
c97eebf2b227: Layer already exists 
284a6c64b82c: Layer already exists 
388eedeb736e: Layer already exists 
2feece0964b8: Layer already exists 
e8b689711f21: Layer already exists 
latest: digest: sha256:dc134f65bec40ed4ea7049188fe1e3915b8e6c3fd999b17effe8ec24868b979c size: 1793

https://hub.docker.com/layers/datasetteproject/datasette/latest/images/sha256-dc134f65bec40ed4ea7049188fe1e3915b8e6c3fd999b17effe8ec24868b979c?context=explore

@simonw simonw closed this as completed Oct 24, 2021
@simonw
Copy link
Owner Author

simonw commented Oct 28, 2021

Wrote about this in my weeknotes: https://simonwillison.net/2021/Oct/28/weeknotes-kubernetes-web-components/

simonw added a commit that referenced this issue Nov 14, 2021
@adamalton
Copy link

You may have just been talking to yourself here, but I found your documentation of the process incredibly useful when I hit the same error myself. Thanks! 🌈

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug ci docker The official Docker image, plus other things related to running Datasette on Docker
Projects
None yet
Development

No branches or pull requests

2 participants