aptly mirror update ERROR: unable to update: download errors #26

Closed
erickeller opened this Issue Apr 10, 2014 · 7 comments

Comments

Projects
None yet
2 participants
@erickeller

Hi there,

by updating my repo mirror the following error occured:
Downloading http://ftp.ch.debian.org/debian/pool/main/liba/libalgorithm-diff-perl/libalgorithm-diff-perl_1.19.02-2.dsc...
ERROR: unable to update: download errors:
unexpected EOF
unexpected EOF

when I restarted the operation, there was nothing in the download queue to be downloaded. How can I be sure the mirror update was successful?

Best Regards

@smira

This comment has been minimized.

Show comment
Hide comment
@smira

smira Apr 10, 2014

Member

Eric, I have no idea what's going on exactly, but a few points:

  1. aptly downloads to temp file always and renames it to final name only when download succeeds
  2. when download queue is built, all files are checked by name and size to match what is expected, I was going to add checksum check as well to be 100% sure

So if aptly mirror update reports empty queue, it should be that all the files have been downloaded. If the error you mentioned is reproducible, could you please send me command sequence?

Member

smira commented Apr 10, 2014

Eric, I have no idea what's going on exactly, but a few points:

  1. aptly downloads to temp file always and renames it to final name only when download succeeds
  2. when download queue is built, all files are checked by name and size to match what is expected, I was going to add checksum check as well to be 100% sure

So if aptly mirror update reports empty queue, it should be that all the files have been downloaded. If the error you mentioned is reproducible, could you please send me command sequence?

@smira

This comment has been minimized.

Show comment
Hide comment
@smira

smira Apr 10, 2014

Member

And one more point: aptly reports all the errors encountered during download in the end.

I'll take a look what could have been causing that.

Member

smira commented Apr 10, 2014

And one more point: aptly reports all the errors encountered during download in the end.

I'll take a look what could have been causing that.

@erickeller

This comment has been minimized.

Show comment
Hide comment
@erickeller

erickeller Apr 10, 2014

Hi Andrey,

thanks for the information, unfortunately I could not reproduce this error... But if you tell me that the queue list is build up accordingly it's enough for me...

best regards

Hi Andrey,

thanks for the information, unfortunately I could not reproduce this error... But if you tell me that the queue list is build up accordingly it's enough for me...

best regards

@erickeller

This comment has been minimized.

Show comment
Hide comment
@erickeller

erickeller Jul 29, 2014

Hi Andrey,

I was able to reproduce the same behavior with a jessie repository. I am not having the details since I did not log the complete process of downloading 50 Go. But I could notice at the end of the aptly update mirror jessie-main:

ERROR: unable to update: download errors:
unexpected EOF
unexpected EOF
unexpected EOF
unexpected EOF

another iteration downloaded 4 packages:

repo@ubu1204repo:/usr/local/repo/aptly$ aptly mirror update jessie-main
Downloading http://ftp.ch.debian.org/debian/dists/jessie/InRelease...
gpgv: Signature made Tue 29 Jul 2014 05:11:17 PM CEST using RSA key ID 46925553
gpgv: Good signature from "Debian Archive Automatic Signing Key (7.0/wheezy) ftpmaster@debian.org"
Downloading & parsing package files...
Downloading http://ftp.ch.debian.org/debian/dists/jessie/main/binary-amd64/Packages.bz2...
Downloading http://ftp.ch.debian.org/debian/dists/jessie/main/binary-amd64/Packages.gz...
Downloading http://ftp.ch.debian.org/debian/dists/jessie/main/binary-i386/Packages.bz2...
Downloading http://ftp.ch.debian.org/debian/dists/jessie/main/binary-i386/Packages.gz...
Downloading http://ftp.ch.debian.org/debian/dists/jessie/main/source/Sources.bz2...
Downloading http://ftp.ch.debian.org/debian/dists/jessie/main/source/Sources.gz...
Building download queue...
Download queue: 4 items (404.99 KiB)
Downloading http://ftp.ch.debian.org/debian/pool/main/m/mplayerthumbs/mplayerthumbs_4.13.1-1_i386.deb...
Downloading http://ftp.ch.debian.org/debian/pool/main/a/alsa-tools/ld10k1_1.0.28-1_amd64.deb...
Downloading http://ftp.ch.debian.org/debian/pool/main/q/qtmultimedia-opensource-src/libqt5multimedia5_5.3.1-2_i386.deb...
Downloading http://ftp.ch.debian.org/debian/pool/main/t/tryton-modules-analytic-sale/tryton-modules-analytic-sale_3.2.0.orig.tar.gz...

Mirror jessie-main has been successfully updated.
repo@ubu1204repo:/usr/local/repo/aptly$ echo $?
0

I did not check if the return code of aptly when the unexpected EOF message occurred. But I guess/hope it's not 0.

Hi Andrey,

I was able to reproduce the same behavior with a jessie repository. I am not having the details since I did not log the complete process of downloading 50 Go. But I could notice at the end of the aptly update mirror jessie-main:

ERROR: unable to update: download errors:
unexpected EOF
unexpected EOF
unexpected EOF
unexpected EOF

another iteration downloaded 4 packages:

repo@ubu1204repo:/usr/local/repo/aptly$ aptly mirror update jessie-main
Downloading http://ftp.ch.debian.org/debian/dists/jessie/InRelease...
gpgv: Signature made Tue 29 Jul 2014 05:11:17 PM CEST using RSA key ID 46925553
gpgv: Good signature from "Debian Archive Automatic Signing Key (7.0/wheezy) ftpmaster@debian.org"
Downloading & parsing package files...
Downloading http://ftp.ch.debian.org/debian/dists/jessie/main/binary-amd64/Packages.bz2...
Downloading http://ftp.ch.debian.org/debian/dists/jessie/main/binary-amd64/Packages.gz...
Downloading http://ftp.ch.debian.org/debian/dists/jessie/main/binary-i386/Packages.bz2...
Downloading http://ftp.ch.debian.org/debian/dists/jessie/main/binary-i386/Packages.gz...
Downloading http://ftp.ch.debian.org/debian/dists/jessie/main/source/Sources.bz2...
Downloading http://ftp.ch.debian.org/debian/dists/jessie/main/source/Sources.gz...
Building download queue...
Download queue: 4 items (404.99 KiB)
Downloading http://ftp.ch.debian.org/debian/pool/main/m/mplayerthumbs/mplayerthumbs_4.13.1-1_i386.deb...
Downloading http://ftp.ch.debian.org/debian/pool/main/a/alsa-tools/ld10k1_1.0.28-1_amd64.deb...
Downloading http://ftp.ch.debian.org/debian/pool/main/q/qtmultimedia-opensource-src/libqt5multimedia5_5.3.1-2_i386.deb...
Downloading http://ftp.ch.debian.org/debian/pool/main/t/tryton-modules-analytic-sale/tryton-modules-analytic-sale_3.2.0.orig.tar.gz...

Mirror jessie-main has been successfully updated.
repo@ubu1204repo:/usr/local/repo/aptly$ echo $?
0

I did not check if the return code of aptly when the unexpected EOF message occurred. But I guess/hope it's not 0.

@smira

This comment has been minimized.

Show comment
Hide comment
@smira

smira Aug 5, 2014

Member

I believe this is more or less correct: you had 4 errors during download (the error message should be improved). On next attempt to run mirror update you had errored files being re-downloaded.

Member

smira commented Aug 5, 2014

I believe this is more or less correct: you had 4 errors during download (the error message should be improved). On next attempt to run mirror update you had errored files being re-downloaded.

@erickeller

This comment has been minimized.

Show comment
Hide comment
@erickeller

erickeller Aug 5, 2014

Hi Andrey,

I agree the error message could be improved, I also was able to reproduce it as hoped, aptly does not return 0 in case of update errors.
I think I will have to put some checker around my update script in order to get a consistent state of the repository before making a snapshot.

Regards

Hi Andrey,

I agree the error message could be improved, I also was able to reproduce it as hoped, aptly does not return 0 in case of update errors.
I think I will have to put some checker around my update script in order to get a consistent state of the repository before making a snapshot.

Regards

@smira smira added the bug label Oct 1, 2014

@smira smira added this to the v0.8 milestone Oct 1, 2014

smira added a commit that referenced this issue Oct 2, 2014

@smira

This comment has been minimized.

Show comment
Hide comment
@smira

smira Oct 2, 2014

Member

@erickeller I've checked once more: aptly returns error code on errors during mirror update phase.

Member

smira commented Oct 2, 2014

@erickeller I've checked once more: aptly returns error code on errors during mirror update phase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment