Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

oneliner patching for packagecloud.io trust issue #17

Closed
wants to merge 7 commits into from

Conversation

fgeorgatos
Copy link
Contributor

@fgeorgatos fgeorgatos commented Mar 1, 2019

@erbou : do you think this patch makes sense?

This is what it fixes: https://renkulab.io/gitlab/damien.bouffard/datalakes/issues/26

@fgeorgatos fgeorgatos self-assigned this Mar 1, 2019
@@ -47,6 +47,7 @@ RUN apt-get update && apt-get install -yq --no-install-recommends \
RUN ln -s /usr/lib/x86_64-linux-musl/libc.so /lib/libc.musl-x86_64.so.1

# install git-lfs
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 6B05F25D762E3157
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you combine this with the line below into a single RUN?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@rokroskar: note that git-lfs is now in the 2.7.x range, should we update that, too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rokroskar : so, let's switch to 2.7.1, no?

@rokroskar rokroskar added this to the 0.3.1 milestone Mar 1, 2019
docker/base/Dockerfile Outdated Show resolved Hide resolved
docker/base/Dockerfile Outdated Show resolved Hide resolved
@rokroskar
Copy link
Member

rokroskar commented Mar 5, 2019

@fgeorgatos I still don't understand which problem this addresses. For example, I can do:

docker run --rm -ti renku/singleuser:latest bash
jovyan@e444698984ce:~$ sudo apt-get update
Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:4 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [691 kB]
Get:7 http://security.ubuntu.com/ubuntu bionic-security/universe Sources [40.2 kB]
Get:8 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [157 kB]
Hit:6 https://packagecloud.io/github/git-lfs/ubuntu bionic InRelease
Get:9 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [353 kB]
Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [943 kB]
Fetched 2,435 kB in 6s (378 kB/s)
Reading package lists... Done
jovyan@e444698984ce:~$

Note that git-lfs was updated without a problem. So where does the issue with the token come from?

jirikuncar and others added 3 commits March 6, 2019 14:28
Co-Authored-By: fgeorgatos <kefalonia@gmail.com>
Co-Authored-By: fgeorgatos <kefalonia@gmail.com>
@rokroskar rokroskar removed this from the 0.4.0 milestone Mar 6, 2019
@rokroskar
Copy link
Member

I can't reproduce this -- keeping the PR open in case it comes back and we can do a minor release.

@erbou
Copy link

erbou commented Mar 12, 2019

Using apt-get update on a Dockerfile based on an image that has the old gpg key for git-lfs is failing because of the expired key, and I believe you have a warning if you are using apt update.

We have the error in images based on 0.3.2 and earlier, but not in latest.

Recommendation is to update the project to use the new images. The risk is that it may go against reproducibility.

You can verify this with the Dockerfile using different combinations of singleuser, or singleuser-r and 0.3.2 or latest.

WORKS:

FROM renku/singleuser-r:latest

USER root

RUN apt-key list && apt-get update

Log shows that it is using the updated public key from packagecloud.io, 6D39 8DBD 30DD 7894 1E2C 4797 FE2A 5F8B DC28 2033.

Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM renku/singleuser-r:latest
 ---> 75edbc518c01
Step 2/3 : USER root
 ---> Using cache
 ---> 3bb102331fe6
Step 3/3 : RUN apt-key list && apt-get update
 ---> Running in 57d7b3fc5fd6
Warning: apt-key output should not be parsed (stdout is not a terminal)
/etc/apt/trusted.gpg
--------------------
pub   rsa4096 2018-10-08 [SCEA]
      6D39 8DBD 30DD 7894 1E2C  4797 FE2A 5F8B DC28 2033
uid           [ unknown] https://packagecloud.io/github/git-lfs (https://packagecloud.io/docs#gpg_signing) <support@packagecloud.io>
sub   rsa4096 2018-10-08 [SEA]

/etc/apt/trusted.gpg.d/ubuntu-keyring-2012-archive.gpg
------------------------------------------------------
pub   rsa4096 2012-05-11 [SC]
      790B C727 7767 219C 42C8  6F93 3B4F E6AC C0B2 1F32
uid           [ unknown] Ubuntu Archive Automatic Signing Key (2012) <ftpmaster@ubuntu.com>

/etc/apt/trusted.gpg.d/ubuntu-keyring-2012-cdimage.gpg
------------------------------------------------------
pub   rsa4096 2012-05-11 [SC]
      8439 38DF 228D 22F7 B374  2BC0 D94A A3F0 EFE2 1092
uid           [ unknown] Ubuntu CD Image Automatic Signing Key (2012) <cdimage@ubuntu.com>

Get:1 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:3 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic/universe Sources [11.5 MB]
Get:7 http://security.ubuntu.com/ubuntu bionic-security/universe Sources [40.2 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic/restricted amd64 Packages [13.5 kB]
Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
Get:10 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [157 kB]
Get:6 https://packagecloud.io/github/git-lfs/ubuntu bionic InRelease [23.2 kB]
Get:11 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [358 kB]
Get:12 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [5,436 B]
Get:13 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [3,910 B]
Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1,344 kB]
Get:15 http://archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [186 kB]
Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/universe Sources [186 kB]
Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [6,966 B]
Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [10.8 kB]
Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [947 kB]
Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [717 kB]
Get:21 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [3,659 B]
Get:22 https://packagecloud.io/github/git-lfs/ubuntu bionic/main amd64 Packages [2,200 B]
Fetched 27.4 MB in 4s (6,748 kB/s)
Reading package lists...
Removing intermediate container 57d7b3fc5fd6
 ---> d7d8dea332ed
Successfully built d7d8dea332ed

FAILS:

FROM renku/singleuser-r:0.3.2

USER root

RUN apt-key list && apt-get update

Log shows that it is using the older public key from packagecloud.io, 418A 7F2F B0E1 E6E7 EABF 6FE8 C2E7 3424 D590 97AB

Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM renku/singleuser-r:0.3.2
 ---> e426283774e7
Step 2/3 : USER root
 ---> Running in ca6efd4942d1
Removing intermediate container ca6efd4942d1
 ---> 935840415007
Step 3/3 : RUN apt-key list && apt-get update
 ---> Running in 46af56bec457
Warning: apt-key output should not be parsed (stdout is not a terminal)
/etc/apt/trusted.gpg
--------------------
pub   rsa4096 2014-01-13 [SCEA] [expired: 2019-01-12]
      418A 7F2F B0E1 E6E7 EABF  6FE8 C2E7 3424 D590 97AB
uid           [ expired] packagecloud ops (production key) <ops@packagecloud.io>

/etc/apt/trusted.gpg.d/ubuntu-keyring-2012-archive.gpg
------------------------------------------------------
pub   rsa4096 2012-05-11 [SC]
      790B C727 7767 219C 42C8  6F93 3B4F E6AC C0B2 1F32
uid           [ unknown] Ubuntu Archive Automatic Signing Key (2012) <ftpmaster@ubuntu.com>

/etc/apt/trusted.gpg.d/ubuntu-keyring-2012-cdimage.gpg
------------------------------------------------------
pub   rsa4096 2012-05-11 [SC]
      8439 38DF 228D 22F7 B374  2BC0 D94A A3F0 EFE2 1092
uid           [ unknown] Ubuntu CD Image Automatic Signing Key (2012) <cdimage@ubuntu.com>

Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:5 http://security.ubuntu.com/ubuntu bionic-security/universe Sources [40.2 kB]
Get:6 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [358 kB]
Get:7 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [5,436 B]
Get:8 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [157 kB]
Get:9 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [3,910 B]
Get:10 http://archive.ubuntu.com/ubuntu bionic/universe Sources [11.5 MB]
Get:11 https://packagecloud.io/github/git-lfs/ubuntu bionic InRelease [23.2 kB]
Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
Get:13 http://archive.ubuntu.com/ubuntu bionic/restricted amd64 Packages [13.5 kB]
Get:14 http://archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [186 kB]
Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1,344 kB]
Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/universe Sources [186 kB]
Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [947 kB]
Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [6,966 B]
Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [717 kB]
Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [10.8 kB]
Get:21 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [3,659 B]
Err:11 https://packagecloud.io/github/git-lfs/ubuntu bionic InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 6B05F25D762E3157
Reading package lists...
W: GPG error: https://packagecloud.io/github/git-lfs/ubuntu bionic InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 6B05F25D762E3157
E: The repository 'https://packagecloud.io/github/git-lfs/ubuntu bionic InRelease' is not signed.
The command '/bin/sh -c apt-key list && apt-get update' returned a non-zero code: 100

@rokroskar
Copy link
Member

rokroskar commented Mar 12, 2019

What you can do is

  1. create a 0.3.2 maintenance branch from the 0.3.2 tag, make a new build and tag it appropriately

Or

  1. push a new 0.3.2 image if the fix seems to have been fixed in the dependency itself

Note that when you make a new tag and push it, the images should be built by Travis automatically.

@fgeorgatos
Copy link
Contributor Author

@rokroskar : I understand the motivation of the maintenance release on 0.3.2, let's do that if we feel this is a recurring need. And, indeed, for reproducibility aims, I'd question our dependency on this one:
curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh
Since it could affect installed git-lfs, I'd like to check with @erbou beforehand about it, once more!

@rokroskar
Copy link
Member

@fgeorgatos note that there's no need to change the dockerfile to fix this, but the base image needs to be rebuilt to use updated apt.

Also, if you want to install git-lfs differently, please propose a change!

@rokroskar
Copy link
Member

I'm closing this for now - please reopen if it becomes an issue again.

@rokroskar rokroskar closed this Mar 28, 2019
@fgeorgatos fgeorgatos deleted the fgeorgatos-packagecloud.io branch March 30, 2019 01:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants