Redis 7.2.0 image missing, build fails

[yossigo]

The build log seems to fail due to an invalid sha256 check, but the value printed seems to be correct:

+ wget -O redis.tar.gz http://download.redis.io/releases/redis-7.2.0.tar.gz
--2023-08-16 09:04:32--  http://download.redis.io/releases/redis-7.2.0.tar.gz
Resolving download.redis.io (download.redis.io)... 45.60.121.1
Connecting to download.redis.io (download.redis.io)|45.60.121.1|:80... connected.
[...]
2023-08-16 09:04:33 (13.9 MB/s) - 'redis.tar.gz' saved [3381269/3381269]

+ echo 8b12e242647635b419a0e1833eda02b65bf64e39eb9e509d9db4888fb3124943 *redis.tar.gz
+ sha256sum -c -
redis.tar.gz: FAILED
sha256sum: WARNING: 1 computed checksum did NOT match

[nsteinmetz]

Hi there,

Looking at http://download.redis.io/ and especially http://download.redis.io/redis-stable.tar.gz.SHA256SUM

The checksum is:

57dbd35f42b40143ea64ec5ae96496ac70730e6f3341c78871d7209f8b5a15d2  redis-stable.tar.gz

which is not the same as in the pipeline 🤔

[yosifkit]

The hash we have comes from https://github.com/redis/redis-hashes. It seems that it does not agree with the release.

Edit:

The redis-stable.tar.gz is repackaged so that it always uncompresses into the redis-stable directory instead of including the version number in the directory

So the redis-stable.tar.gz.SHA256SUM would not be correct for 7.2.0 anyway.

[yosifkit]

Might be a network or CDN problem? It worked fine in a local, interactive test. I'll keep digging so we can get the update in.

root@b3a421e69f10:/# wget 'http://download.redis.io/releases/redis-7.2.0.tar.gz'
--2023-08-16 19:14:09--  http://download.redis.io/releases/redis-7.2.0.tar.gz
Resolving download.redis.io (download.redis.io)... 45.60.121.1
Connecting to download.redis.io (download.redis.io)|45.60.121.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3381269 (3.2M) [application/octet-stream]
Saving to: 'redis-7.2.0.tar.gz'

redis-7.2.0.tar.gz  100%[===================>]   3.22M  4.57MB/s    in 0.7s

2023-08-16 19:14:10 (4.57 MB/s) - 'redis-7.2.0.tar.gz' saved [3381269/3381269]

root@b3a421e69f10:/# echo 8b12e242647635b419a0e1833eda02b65bf64e39eb9e509d9db4888fb3124943 *redis*.tar.gz | sha256sum -c -
redis-7.2.0.tar.gz: OK
root@b3a421e69f10:/#

[nsteinmetz]

Weird:

echo 57dbd35f42b40143ea64ec5ae96496ac70730e6f3341c78871d7209f8b5a15d2 redis-stable.tar.gz  | sha256sum -c -
redis-stable.tar.gz: OK

Maybe package differs somehow from the 7.2.0 release.

Otherwise, it also works from my side:

wget 'http://download.redis.io/releases/redis-7.2.0.tar.gz'
--2023-08-16 21:23:12--  http://download.redis.io/releases/redis-7.2.0.tar.gz
Résolution de download.redis.io (download.redis.io)… 45.60.123.1
Connexion à download.redis.io (download.redis.io)|45.60.123.1|:80… connecté.
requête HTTP transmise, en attente de la réponse… 200 OK
Taille : 3381269 (3,2M) [application/octet-stream]
Sauvegarde en : « redis-7.2.0.tar.gz »

redis-7.2.0.tar.gz                                         100%[======================================================================================================================================>]   3,22M  16,1MB/s    ds 0,2s

2023-08-16 21:23:12 (16,1 MB/s) — « redis-7.2.0.tar.gz » sauvegardé [3381269/3381269]

echo 8b12e242647635b419a0e1833eda02b65bf64e39eb9e509d9db4888fb3124943 redis-7.2.0.tar.gz | sha256sum -c -
redis-7.2.0.tar.gz: OK

So maybe someone just need to click on the "Retry" button on Jenkins 😉

[yosifkit]

Jenkins retried a few times already and I needed to move 7.2 to a non-rc directory and make it latest, so PR is open to have GHA test the build: #374.

[nsteinmetz]

Thanks @yosifkit :)

[yossigo]

Thank you @yosifkit. Was this a result of redis-7.2.0 not identical to redis-stable, or something else? I'm still puzzled about why the same flow failed on CI and succeeded externally. We have to clean up some stuff on the Redis side anyway, but I'm trying to figure out if that mess was a contributing factor.

[yosifkit]

I'm not certain why our server running update.sh and doing the test build is getting the wrong file. It runs in AWS. I've just run an interactive test on the server and it still downloads an incorrect file for redis-7.2.0.tar.gz with both curl and wget; perhaps a CDN cache bug somewhere? It's a bit strange since I get the same IP and Etag when I test locally, but I get the correct file.

$ docker run -it --rm buildpack-deps:bookworm
root@e6c4b9fb8a8d:/# wget -O redis.tar.gz http://download.redis.io/releases/redis-7.2.0.tar.gz
--2023-08-17 16:37:29--  http://download.redis.io/releases/redis-7.2.0.tar.gz
Resolving download.redis.io (download.redis.io)... 45.60.121.1
Connecting to download.redis.io (download.redis.io)|45.60.121.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3381269 (3.2M) [application/octet-stream]
Saving to: 'redis.tar.gz'

redis.tar.gz        100%[===================>]   3.22M  --.-KB/s    in 0.08s

2023-08-17 16:37:29 (39.9 MB/s) - 'redis.tar.gz' saved [3381269/3381269]

root@e6c4b9fb8a8d:/# sha256sum redis.tar.gz
089eacdec00aa75bdafb15e9bd54448f0a3dedf194000407cff9742fd2ddaf8d  redis.tar.gz
root@e6c4b9fb8a8d:/# curl -fL -vv http://download.redis.io/releases/redis-7.2.0.tar.gz | sha256sum
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 45.60.121.1:80...
* Connected to download.redis.io (45.60.121.1) port 80 (#0)
> GET /releases/redis-7.2.0.tar.gz HTTP/1.1
> Host: download.redis.io
> User-Agent: curl/7.88.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Etag: "64db4858-339815"
< Last-Modified: Tue, 15 Aug 2023 09:41:44 GMT
< Content-Type: application/octet-stream
< Content-Length: 3381269
< Cache-Control: max-age=48, public
< Expires: Thu, 17 Aug 2023 16:45:14 GMT
< Date: Thu, 17 Aug 2023 16:44:26 GMT
< Set-Cookie: visid_incap_2333563=BlmMIs5+TFCRNFOyj0mju2pO3mQAAAAAQUIPAAAAAACIWgkEn1Bs5cEhKhtFzNYC; expires=Fri, 16 Aug 2024 06:26:15 GMT; HttpOnly; path=/; Domain=.redis.io
< Set-Cookie: incap_ses_2107_2333563=a/jPcJc4vzCVrQ3CtZE9HWpO3mQAAAAAICTb9FEH4aZTX3udF7/VxA==; path=/; Domain=.redis.io
< X-CDN: Imperva
< X-Iinfo: 18-136792554-136722443 3CNN RT(1692290666249 0) q(0 0 0 -1) r(0 0)
<
{ [13032 bytes data]
100 3302k  100 3302k    0     0  1945k      0  0:00:01  0:00:01 --:--:-- 1945k
* Connection #0 to host download.redis.io left intact
089eacdec00aa75bdafb15e9bd54448f0a3dedf194000407cff9742fd2ddaf8d  -

Trying to extract the file shows that it is corrupted:

root@e6c4b9fb8a8d:/# tar -xvf redis.tar.gz
redis-7.2.0/ 
...
redis-7.2.0/utils/releasetools/01_create_tarball.sh

gzip: stdin: invalid compressed data--crc error
redis-7.2.0/utils/releasetools/02_upload_tarball.sh
...
tar: Child returned status 1
tar: Error is not recoverable: exiting now