Skip to content

Crenwuste/Parallel-Downloader

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Parallel File Downloader (Java)

Project Structure

  • Main.java: entry point that triggers a download.
  • FileDownloader.java: orchestrates the download (HEAD request, chunk calculation, scheduling parallel downloads, combining results).
  • ChunkDownloader.java: Callable<byte[]> that downloads one specific byte range.
  • FileDownloaderTest.java: unit tests for file size detection, chunk calculation, and chunk combination.

How the Downloader Works

  1. Discover file size (HEAD request)
    FileDownloader#getFileSize sends an HTTP HEAD request to the provided URL and reads the Content-Length header to determine the total size of the file in bytes. The server should return Accept-Ranges: bytes, indicating support for range requests.

  2. Split into chunks
    Based on the file size and configured number of threads, FileDownloader#calculateChunks divides the file into contiguous ranges [start, end] so that every byte in the file is covered exactly once and there are no gaps or overlaps.

  3. Download chunks in parallel
    FileDownloader#download creates a fixed-size thread pool and submits a ChunkDownloader task for each range. Each ChunkDownloader:

    • sends an HTTP GET request to the same URL
    • adds the header Range: bytes=startByte-endByte
    • returns the response body as a byte[]
  4. Combine chunks into the final file
    Once all futures complete, FileDownloader#combineChunks concatenates the byte[] results in the correct order and writes the final byte array to the output file on disk.

Running Unit Tests

The project includes unit tests that validate:

  • reading Content-Length from a HEAD response (getFileSize)
  • splitting a file size into correct chunks (calculateChunks)
  • correct concatenation of byte arrays (combineChunks)
  • error handling when the server returns a non-200 status

About

Java parallel file downloader that uses HTTP range requests to split files into chunks, download them concurrently, and merge them into a final output.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages