Force install dependencies in old Debian with expired keys #34922

rdner · 2023-03-24T12:56:05Z

What does this PR do?

This change is the same as #33915 but for the rest of the Beats.

Why is it important?

We cannot build snapshots right now:

06:29:52 Status: Downloaded newer image for docker.elastic.co/beats-dev/golang-crossbuild:1.18.10-main-debian8
06:29:58 >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=386, GOARM=, GOOS=linux, PLATFORM_ID=linux-386]
06:29:58 >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
06:30:05 W: GPG error: http://archive.debian.org/ jessie Release: The following signatures were invalid: KEYEXPIRED 1587841717
06:30:05 W: Failed to fetch http://security.debian.org/debian-security/dists/jessie/updates/main/binary-amd64/Packages  404  Not Found [IP: 151.101.194.132 80]
06:30:05 
06:30:05 W: Failed to fetch http://security.debian.org/debian-security/dists/jessie/updates/main/binary-i386/Packages  404  Not Found [IP: 151.101.194.132 80]
06:30:05 
06:30:05 E: Some index files failed to download. They have been ignored, or old ones used instead.
06:30:05 Error: running "apt-get update" failed with exit code 100
06:30:05 Error: failed building for linux/386: exit status 100
06:30:05 failed building for linux/386: exit status 100
06:30:05 W: GPG error: http://archive.debian.org/ jessie Release: The following signatures were invalid: KEYEXPIRED 1587841717
06:30:05 W: Failed to fetch http://security.debian.org/debian-security/dists/jessie/updates/main/binary-amd64/Packages  404  Not Found [IP: 151.101.66.132 80]
06:30:05 
06:30:05 W: Failed to fetch http://security.debian.org/debian-security/dists/jessie/updates/main/binary-i386/Packages  404  Not Found [IP: 151.101.66.132 80]
06:30:05 
06:30:05 E: Some index files failed to download. They have been ignored, or old ones used instead.
06:30:05 Error: running "apt-get update" failed with exit code 100
06:30:05 Error: failed building for linux/amd64: exit status 100
06:30:05 failed building for linux/amd64: exit status 100

https://internal-ci.elastic.co/job/elastic+unified-release+master+snapshot-multijob-7.17/361/display/redirect

How to test locally

mage package

elasticmachine · 2023-03-24T12:56:10Z

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

rdner · 2023-03-24T13:02:01Z

elasticsearch-ci/docs failure is a known issue related to https://github.blog/2023-03-23-we-updated-our-rsa-ssh-host-key/

elasticmachine · 2023-03-24T14:04:10Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-03-27T07:33:00.365+0000
Duration: 92 min 56 sec

Test stats 🧪

Test	Results
Failed	0
Passed	22266
Skipped	1572
Total	23838

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

rdner · 2023-03-24T16:35:06Z

It's broken, I'm trying out more options.

rdner · 2023-03-24T16:49:57Z

I have these errors when I just run apt update on raw Debian Jessie (8):

root@3b6c3be95753:/etc/apt# apt update
Ign http://security.debian.org jessie/updates InRelease
Ign http://security.debian.org jessie/updates Release.gpg
Ign http://security.debian.org jessie/updates Release 
Ign http://ftp.us.debian.org jessie InRelease
Ign http://ftp.us.debian.org jessie Release.gpg
Ign http://ftp.us.debian.org jessie Release
Err http://security.debian.org jessie/updates/main armhf Packages
  404  Not Found [IP: 151.101.2.132 80]
Err http://security.debian.org jessie/updates/contrib armhf Packages
  404  Not Found [IP: 151.101.2.132 80]
Err http://security.debian.org jessie/updates/non-free armhf Packages
  404  Not Found [IP: 151.101.2.132 80]
Err http://ftp.us.debian.org jessie/main armhf Packages
  404  Not Found [IP: 208.80.154.139 80]
Err http://ftp.us.debian.org jessie/contrib armhf Packages
  404  Not Found [IP: 208.80.154.139 80]
Err http://ftp.us.debian.org jessie/non-free armhf Packages
  404  Not Found [IP: 208.80.154.139 80]
W: Failed to fetch http://security.debian.org/dists/jessie/updates/main/binary-armhf/Packages  404  Not Found [IP: 151.101.2.132 80]

W: Failed to fetch http://security.debian.org/dists/jessie/updates/contrib/binary-armhf/Packages  404  Not Found [IP: 151.101.2.132 80]

W: Failed to fetch http://security.debian.org/dists/jessie/updates/non-free/binary-armhf/Packages  404  Not Found [IP: 151.101.2.132 80]

W: Failed to fetch http://ftp.us.debian.org/debian/dists/jessie/main/binary-armhf/Packages  404  Not Found [IP: 208.80.154.139 80]

W: Failed to fetch http://ftp.us.debian.org/debian/dists/jessie/contrib/binary-armhf/Packages  404  Not Found [IP: 208.80.154.139 80]

W: Failed to fetch http://ftp.us.debian.org/debian/dists/jessie/non-free/binary-armhf/Packages  404  Not Found [IP: 208.80.154.139 80]

E: Some index files failed to download. They have been ignored, or old ones used instead.

Looks like the official repositories are not available anymore.

I used this to get a shell on Mac M1:

docker run --platform "linux/arm/v7" --rm -it --entrypoint /bin/bash debian:jessie

This change is the same as elastic#33915 but for the rest of the Beats.

rdner · 2023-03-24T18:54:01Z

I fixed the source.list (kind of) but I still have this problem to solve:

The following packages have unmet dependencies:
 libgcrypt20:i386 : Depends: libc6:i386 (>= 2.15) but it is not installable
                    Depends: libgpg-error0:i386 (>= 1.14) but it is not going to be installed
 libsystemd0:i386 : PreDepends: libc6:i386 (>= 2.17) but it is not installable
                    PreDepends: libgcc1:i386 (>= 1:4.1.1) but it is not installable
                    PreDepends: liblzma5:i386 (>= 5.1.1alpha+20120614) but it is not installable
                    PreDepends: libselinux1:i386 (>= 1.32) but it is not installable
E: Unable to correct problems, you have held broken packages.

dev-tools/mage/pkgdeps.go

rdner · 2023-03-27T07:36:50Z

Even after fixing the repositories our crossbuild cannot install packages, the dependencies cannot be resolved in the new archive repository:

rdner · 2023-03-28T11:20:30Z

This should be fixed by backporting #34939 to 7.17

rdner added Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team backport-v7.17.0 Automated backport with mergify backport-v8.7.0 Automated backport with mergify labels Mar 24, 2023

rdner requested a review from a team as a code owner March 24, 2023 12:56

rdner self-assigned this Mar 24, 2023

rdner requested review from cmacknz and leehinman and removed request for a team March 24, 2023 12:56

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Mar 24, 2023

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Mar 24, 2023

rdner mentioned this pull request Mar 24, 2023

Stop crossbuilding using Debian 8 #34921

Merged

cmacknz approved these changes Mar 24, 2023

View reviewed changes

rdner marked this pull request as draft March 24, 2023 16:34

rdner force-pushed the force-install-old-debian branch from d762a69 to da0b8ae Compare March 24, 2023 16:51

rdner changed the base branch from main to 7.17 March 24, 2023 16:53

rdner removed backport-v7.17.0 Automated backport with mergify backport-v8.7.0 Automated backport with mergify labels Mar 24, 2023

Force install dependencies in old Debian with expired keys

e407fc1

This change is the same as elastic#33915 but for the rest of the Beats.

rdner force-pushed the force-install-old-debian branch from da0b8ae to e407fc1 Compare March 24, 2023 17:06

faec reviewed Mar 24, 2023

View reviewed changes

dev-tools/mage/pkgdeps.go Outdated Show resolved Hide resolved

Add repository replacement script

dbf0c3a

rdner force-pushed the force-install-old-debian branch from dd39118 to dbf0c3a Compare March 27, 2023 07:32

rdner closed this Mar 28, 2023

rdner mentioned this pull request Apr 18, 2023

[Automation] Bump Golang version to 1.19.8 elastic/golang-crossbuild#282

Merged

v1v mentioned this pull request Apr 18, 2023

Debian 8 errors when installing packages elastic/golang-crossbuild#283

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Force install dependencies in old Debian with expired keys #34922

Force install dependencies in old Debian with expired keys #34922

rdner commented Mar 24, 2023 •

edited

elasticmachine commented Mar 24, 2023

rdner commented Mar 24, 2023

elasticmachine commented Mar 24, 2023 •

edited by jenkins-beats-ci bot

Build stats

Test stats 🧪

rdner commented Mar 24, 2023

rdner commented Mar 24, 2023 •

edited

rdner commented Mar 24, 2023

rdner commented Mar 27, 2023 •

edited

rdner commented Mar 28, 2023 •

edited

Force install dependencies in old Debian with expired keys #34922

Force install dependencies in old Debian with expired keys #34922

Conversation

rdner commented Mar 24, 2023 • edited

What does this PR do?

Why is it important?

How to test locally

elasticmachine commented Mar 24, 2023

rdner commented Mar 24, 2023

elasticmachine commented Mar 24, 2023 • edited by jenkins-beats-ci bot

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

rdner commented Mar 24, 2023

rdner commented Mar 24, 2023 • edited

rdner commented Mar 24, 2023

rdner commented Mar 27, 2023 • edited

rdner commented Mar 28, 2023 • edited

rdner commented Mar 24, 2023 •

edited

elasticmachine commented Mar 24, 2023 •

edited by jenkins-beats-ci bot

rdner commented Mar 24, 2023 •

edited

rdner commented Mar 27, 2023 •

edited

rdner commented Mar 28, 2023 •

edited