Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support HTTP range requests #7

Closed
nroi opened this issue Apr 12, 2020 · 0 comments
Closed

Support HTTP range requests #7

nroi opened this issue Apr 12, 2020 · 0 comments
Labels
go-live Required change before the first release

Comments

@nroi
Copy link
Owner

nroi commented Apr 12, 2020

HTTP range requests are used by pacman if the file is already partially available on the local file system, so that pacman does not download the entire file.

Notice that there is one special case which complicates matters:
Assume that the client sends a request to download a file starting at byte b. Assume that flexo is not able to fulfill this request from the local cache because we have only s bytes of this file available locally, where s < b. So in that case, we need to fetch the file from a remote mirror. What range are we going to request from the remote mirror?

  1. Requesting the range s- (i.e., everything starting from byte s) would allow us to concatenate the payload to the existing file. This would not work as intended if b is much larger than s: Imagine the client has already 1GiB of the file available locally, but flexo does not have this file available locally yet. In this case, flexo would first have to download 1GiB from the remote mirror until it could start sending data to the client. With a slow internet connection, the client could time out before he receives the first byte.
  2. Requesting the range b- would allow us to start sending data to the client immediately, so we would not run into timeouts or anything similar. But the remaining question is: how should we store the payload we receive from the server? Flexo is currently based on the assumption that all locally stored files start at byte 0, and not some arbitrary byte b. Storing incomplete files that lack data at their beginning (not their end) would make things much more difficult.

Since we assume that the case described above is a rare special case, we choose the following workaround:
Instead of sending a request to the remote mirror, we just respond with a redirect so that the client fetches the file directly from the remote mirror.

@nroi nroi added the go-live Required change before the first release label Apr 12, 2020
@nroi nroi closed this as completed in f66e647 Apr 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
go-live Required change before the first release
Projects
None yet
Development

No branches or pull requests

1 participant