Skip to content
This repository has been archived by the owner on Mar 27, 2018. It is now read-only.

apt-get update 404 on http://httpredir.debian.org/debian/dists/jessie/main/binary-amd64/Packages #31

Closed
bneijt opened this issue Oct 18, 2015 · 22 comments

Comments

@bneijt
Copy link

bneijt commented Oct 18, 2015

It seems that running apt-get update fails with 404 Not Found. The referenced Packages file can be downloaded as Packages.gz and Packages.xz on the host, but not as plain Packages.

Expected behavior: the newest version of the package index is downloaded
Actual behavior: 404 not found.

To reproduce:

$ docker run debian:jessie cat /etc/apt/sources.list
deb http://httpredir.debian.org/debian jessie main
deb http://httpredir.debian.org/debian jessie-updates main
deb http://security.debian.org jessie/updates main

$ docker run debian:jessie apt-get update
Ign http://security.debian.org jessie/updates InRelease
Ign http://httpredir.debian.org jessie InRelease
Ign http://security.debian.org jessie/updates Release.gpg
Ign http://security.debian.org jessie/updates Release
Ign http://httpredir.debian.org jessie-updates InRelease
Err http://security.debian.org jessie/updates/main amd64 Packages

Err http://security.debian.org jessie/updates/main amd64 Packages

Err http://security.debian.org jessie/updates/main amd64 Packages

Err http://security.debian.org jessie/updates/main amd64 Packages

Err http://security.debian.org jessie/updates/main amd64 Packages
  404  Not Found [IP: 195.20.242.89 80]
Ign http://httpredir.debian.org jessie Release.gpg
Ign http://httpredir.debian.org jessie-updates Release.gpg
Ign http://httpredir.debian.org jessie Release
Ign http://httpredir.debian.org jessie-updates Release
Err http://httpredir.debian.org jessie/main amd64 Packages
  404  Not Found [IP: 176.9.184.93 80]
Err http://httpredir.debian.org jessie-updates/main amd64 Packages
  404  Not Found [IP: 176.9.184.93 80]
W: Failed to fetch http://security.debian.org/dists/jessie/updates/main/binary-amd64/Packages  404  Not Found [IP: 195.20.242.89 80]

W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie/main/binary-amd64/Packages  404  Not Found [IP: 176.9.184.93 80]

W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie-updates/main/binary-amd64/Packages  404  Not Found [IP: 176.9.184.93 80]

E: Some index files failed to download. They have been ignored, or old ones used instead.
$ docker --version
Docker version 1.8.3, build f4bf5c7-dirty
@yosifkit
Copy link
Collaborator

It seems that there must've been a server down; it is working here.

@bneijt
Copy link
Author

bneijt commented Oct 19, 2015

Weird, I get
'''
$ curl http://security.debian.org/dists/jessie/updates/main/binary-amd64/Packages

<title>404 Not Found</title>

Not Found

The requested URL /dists/jessie/updates/main/binary-amd64/Packages was not found on this server.


Apache Server at security.debian.org Port 80 '''

Any ideas?

@yosifkit
Copy link
Collaborator

I get the same 404 if I curl that url, but apt-get update works just fine. apt-get actually requests the gzipped version. I tried this: curl --head http://security.debian.org/dists/jessie/updates/main/binary-amd64/Packages.gz

@tianon
Copy link
Owner

tianon commented Oct 19, 2015

Yeah, seems to work just fine here:

$ docker run -it --rm debian:jessie apt-get update
Get:1 http://security.debian.org jessie/updates InRelease [63.1 kB]
Ign http://httpredir.debian.org jessie InRelease          
Get:2 http://security.debian.org jessie/updates/main amd64 Packages [182 kB]
Get:3 http://httpredir.debian.org jessie-updates InRelease [135 kB]
Get:4 http://httpredir.debian.org jessie Release.gpg [2373 B]        
Get:5 http://httpredir.debian.org jessie-updates/main amd64 Packages [3653 B]
Get:6 http://httpredir.debian.org jessie Release [148 kB]
Get:7 http://httpredir.debian.org jessie/main amd64 Packages [9035 kB]
Fetched 9569 kB in 4s (2041 kB/s) 
Reading package lists... Done

@tianon
Copy link
Owner

tianon commented Oct 19, 2015

Sounds like it's definitely a funky mirror hiccup.

@bneijt
Copy link
Author

bneijt commented Oct 20, 2015

Must be something with my network setup then. Thank you all for trying.

@bneijt bneijt closed this as completed Oct 20, 2015
@quasipedia
Copy link

@bneijt - Maybe you fixed it, but I suspect your problem may have been like mine: I set up the proxy to whitelist debian.org, but the gzipped package list redirects with 302 to another mirror next to me. Use curl --head from your host to discover which one or curl -L to see if your host can get to it...

HTH! :)

@bneijt
Copy link
Author

bneijt commented Nov 15, 2015

I had another go and found my problem: it was a firewall setting. The hint that helped me was: HTTP did not work, FTP did work.

The machine running the docker images also had a PREROUTING redirect rule for hosting on port 80 from an unprivileged program:

 Chain PREROUTING (policy ACCEPT)
 num  target     prot opt source               destination         
 1    REDIRECT   tcp  --  anywhere             anywhere             tcp dpt:http redir ports 5000
 2    DOCKER     all  --  anywhere             anywhere             ADDRTYPE match dst-type LOCAL

This was causing HTTP to fail, but checks like ping to succeed. There was also a nice 404 because the local service on port 5000 was responding 404.

Sorry for wasting time here!

@marek-obuchowicz
Copy link

Actually it seems to still happen for me, without any firewall / web proxy:

docker run -it --rm debian:jessie apt-get update
Ign http://httpredir.debian.org jessie InRelease
Get:1 http://security.debian.org jessie/updates InRelease [63.1 kB]
Ign http://httpredir.debian.org jessie-updates InRelease
Ign http://httpredir.debian.org jessie Release.gpg
Ign http://httpredir.debian.org jessie-updates Release.gpg
Ign http://httpredir.debian.org jessie Release
Ign http://httpredir.debian.org jessie-updates Release
Get:2 http://security.debian.org jessie/updates/main amd64 Packages [383 kB]
Err http://httpredir.debian.org jessie/main amd64 Packages
  404  Not Found [IP: 5.153.231.35 80]
Err http://httpredir.debian.org jessie-updates/main amd64 Packages
  404  Not Found [IP: 5.153.231.35 80]
Fetched 447 kB in 1s (356 kB/s)
W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie/main/binary-amd64/Packages  404  Not Found [IP: 5.153.231.35 80]

W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie-updates/main/binary-amd64/Packages  404  Not Found [IP: 5.153.231.35 80]

E: Some index files failed to download. They have been ignored, or old ones used instead.

@engineerakki
Copy link

@marek-obuchowicz

I am facing exact same issue
did you find any solution?

@marek-obuchowicz
Copy link

For me it was docker-machine issue. It did not refresh DNS servers when I moved to another network. Restart of docker-machine has helped.

@tburnett80
Copy link

Its a CDN issue I think,

GET http://deb.debian.org/debian/dists/jessie/InRelease HTTP/1.1
Host: deb.debian.org

responds with
HTTP/1.1 302 Found
Date: Sun, 20 Nov 2016 07:39:37 GMT
Server: Apache
Location: http://cdn-fastly.deb.debian.org/debian/dists/jessie/InRelease
Content-Length: 310
Content-Type: text/html; charset=iso-8859-1

So
GET http://cdn-fastly.deb.debian.org/debian/dists/jessie/InRelease HTTP/1.1
Host: cdn-fastly.deb.debian.org

HTTP/1.1 404 Not Found
Server: Apache
Content-Type: text/html; charset=iso-8859-1
Via: 1.1 varnish
Fastly-Debug-Digest: 9527c5c1499ebb0550ce27502e9d291aae30e5bb3c04a33757ef522c9f1fe819
Content-Length: 291
Accept-Ranges: bytes
Date: Sun, 20 Nov 2016 07:39:38 GMT
Via: 1.1 varnish
Age: 79
Connection: keep-alive
X-Served-By: cache-ams4141-AMS, cache-ord1737-ORD
X-Cache: HIT, HIT
X-Cache-Hits: 1, 2
X-Timer: S1479627578.037931,VS0,VE0

@badbob
Copy link

badbob commented Jan 21, 2017

Have very similar issue on BananaPi single board computer today.

root@bananapi ~ # apt-get update
Hit http://security.debian.org jessie/updates InRelease
Ign http://ftp.debian.org jessie InRelease
Hit http://dl.bananian.org 1604 InRelease
Hit http://ftp.debian.org jessie Release.gpg
Hit http://ftp.debian.org jessie Release
Hit http://security.debian.org jessie/updates/main armhf Packages
Hit http://security.debian.org jessie/updates/main Translation-en
Hit http://dl.bananian.org 1604/main armhf Packages
Get:1 http://ftp.debian.org jessie/main armhf Packages [6,632 kB]
Ign http://dl.bananian.org 1604/main Translation-en_US
Ign http://dl.bananian.org 1604/main Translation-en
Ign http://dl.bananian.org 1604/main Translation-de_DE
Hit http://ftp.debian.org jessie/main Translation-en
Hit http://ftp.debian.org jessie/main Translation-de_DE
Err http://ftp.debian.org jessie/main armhf Packages

Err http://ftp.debian.org jessie/main armhf Packages

Err http://ftp.debian.org jessie/main armhf Packages
  404  Not Found [IP: 130.89.148.12 80]
Fetched 6,632 kB in 9s (695 kB/s)
W: Failed to fetch http://ftp.debian.org/debian/dists/jessie/main/binary-armhf/Packages  404  Not Found [IP: 130.89.148.12 80]

E: Some index files failed to download. They have been ignored, or old ones used instead.
apt-get update  12.56s user 1.77s system 147% cpu 9.742 total

SOLUTION: use aptitude instead of apt.

@thaiduchyk
Copy link

I had the same problem. Here is the solution that helped me: https://robinwinslow.uk/2016/06/23/fix-docker-networking-dns/

@gayathrigcc
Copy link

apt-get update is failing :(
Hit http://software.ligo.org jessie InRelease
Hit http://security.debian.org jessie/updates InRelease
Ign http://ftp.de.debian.org jessie InRelease
Ign http://ftp.us.debian.org jessie InRelease
Hit http://ftp.de.debian.org sid InRelease
Hit http://ftp.us.debian.org jessie Release.gpg
Hit http://security.debian.org jessie/updates/main Sources
Hit http://ftp.de.debian.org jessie Release.gpg
Hit http://ftp.us.debian.org jessie Release
Hit http://software.ligo.org jessie/contrib Sources
Hit http://security.debian.org jessie/updates/main amd64 Packages
Ign http://httpredir.debian.org jessie InRelease
Hit http://security.debian.org jessie/updates/main i386 Packages
Get:1 http://ftp.de.debian.org sid/main amd64 Packages/DiffIndex [27.9 kB]
Hit http://software.ligo.org jessie/contrib amd64 Packages
Get:2 http://ftp.us.debian.org jessie/main amd64 Packages [6,776 kB]
Hit http://httpredir.debian.org jessie-updates InRelease
Hit http://httpredir.debian.org jessie Release.gpg
Get:3 http://ftp.us.debian.org jessie/contrib amd64 Packages [50.2 kB]
Get:4 http://ftp.us.debian.org jessie/non-free amd64 Packages [83.6 kB]
Get:5 http://ftp.de.debian.org sid/main i386 Packages/DiffIndex [27.9 kB]
Get:6 http://ftp.us.debian.org jessie/main i386 Packages [6,779 kB]
Get:7 http://ftp.us.debian.org jessie/contrib i386 Packages [49.5 kB]
Hit http://ftp.de.debian.org jessie Release
Get:8 http://httpredir.debian.org jessie-updates/main Sources [15.4 kB]
Get:9 http://ftp.us.debian.org jessie/non-free i386 Packages [80.4 kB]
Get:10 http://httpredir.debian.org jessie-updates/main amd64 Packages/DiffIndex [6,916 B]
Get:11 http://httpredir.debian.org jessie-updates/main i386 Packages/DiffIndex [6,916 B]
Hit http://httpredir.debian.org jessie Release
Hit http://ftp.de.debian.org jessie/main amd64 Packages
Hit http://ftp.de.debian.org jessie/main i386 Packages
Get:12 http://httpredir.debian.org jessie/main amd64 Packages [6,776 kB]
Hit http://httpredir.debian.org jessie/main Sources
Get:13 http://httpredir.debian.org jessie/main i386 Packages [6,779 kB]
Fetched 69.7 kB in 9s (7,083 B/s)
W: Failed to fetch http://security.debian.org/dists/jessie/updates/InRelease Unable to find expected entry 'main/binary-sid/Packages' in Release file (Wrong sources.list entry or malformed file)

W: Failed to fetch http://software.ligo.org/lscsoft/debian/dists/jessie/InRelease Unable to find expected entry 'contrib/binary-i386/Packages' in Release file (Wrong sources.list entry or malformed file)

W: Failed to fetch http://ftp.de.debian.org/debian/dists/sid/InRelease Unable to find expected entry 'main/binary-sid/Packages' in Release file (Wrong sources.list entry or malformed file)

W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie-updates/InRelease Unable to find expected entry 'main/binary-sid/Packages' in Release file (Wrong sources.list entry or malformed file)

W: Failed to fetch http://ftp.us.debian.org/debian/dists/jessie/Release Unable to find expected entry 'main/binary-sid/Packages' in Release file (Wrong sources.list entry or malformed file)

W: Failed to fetch http://ftp.de.debian.org/debian/dists/jessie/Release Unable to find expected entry 'main/binary-sid/Packages' in Release file (Wrong sources.list entry or malformed file)

W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie/Release Unable to find expected entry 'main/binary-sid/Packages' in Release file (Wrong sources.list entry or malformed file)

E: Some index files failed to download. They have been ignored, or old ones used instead

@AVNadimpalli
Copy link

Please try commenting the below line in /etc/apt/source.list

deb [arch=amd64] https://download.docker.com/linux/debian xenial stable

apt-get update works for me after commenting that line.

@Martlark
Copy link

If you are working behind a proxy you need to tell the docker the proxy settings.

Use this alias to create a docker-build command with proxy settings:

cat << EOF >> ~/.bashrc
export HTTP_PROXY=http://proxy.paranoid.corp:8080
export HTTPS_PROXY=http://proxy.paranoid.corp:8080
alias docker-build="sudo docker build --build-arg http_proxy=$HTTP_PROXY --build-arg https_proxy=$HTTPS_PROXY"
EOF

Then source the new bashrc:

. ~/.bashrc

Of course your proxy will be different. Ask your IT support.

@Pherserk
Copy link

sudo service docker restart && docker build --no-cache=true .

@m0onspell
Copy link

Thanks @Pherserk. I just restarted docker-machine and it worked. Already thought it's something really tough.

@eisterman
Copy link

I have the sane issue, but i don't have Docker...

@blag
Copy link

blag commented Jun 12, 2017

I had this issue, but it was fixed when I rebuilt the image with --no-cache and --pull.

@jeby6372
Copy link

jeby6372 commented Feb 15, 2018

This is a python-acme issue : cerbot (free CA SSL certs). I just faced it tonight on debian strecth .. so same impact on Ubuntu platforms obviously .. does anybody already place a ticket about that (on debian or Ubuntu support platform) ?
Just to circumscribe the problem impact .. tell me which verison you're running on Ubuntu or debian

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests