Fix librepo isn't able to load zchunk files from next server on failure #151
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Zchunk uses a simple state machine in librepo to determine where it's at in the download process (is it downloading the header or the body). The yandex.ru mirror will fail if more than one range is in a request, but fails in a nonstandard way. librepo catches the failure and moves on to the next mirror, but truncates the downloaded zchunk header. This causes the state machine to be out of sync with reality. I've fixed this by only truncating a zchunk header if the state machine is in the header download state, which means that the next mirror will only download any missing chunks. Zchunk will only download missing or corrupted chunks, so this also gives us automatic resume on zchunk metadata downloads. Signed-off-by: Jonathan Dieter <jdieter@gmail.com>
Conan-Kudo
approved these changes
May 12, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Zchunk uses a simple state machine in librepo to determine where it's at
in the download process (is it downloading the header or the body).
The yandex.ru mirror will fail if more than one range is in a request, but
fails in a nonstandard way. librepo catches the failure and moves on to
the next mirror, but truncates the downloaded zchunk header. This causes
the state machine to be out of sync with reality.
I've fixed this by only truncating the zchunk header if the state machine is
in the header download state, which means that the next mirror will only
download any missing chunks. Zchunk will only download missing or
corrupted chunks, so this also gives us automatic resume on zchunk
metadata downloads.
See https://bugzilla.redhat.com/show_bug.cgi?id=1706321 for the initial bug
report. Please pull this into F30, as our Russian users are running into
problems because of this bug.