Threaded Downloader

A high-performance C++ utility for multi-threaded, resumable file downloads. This application optimizes download speeds by partitioning files into segments and downloading them concurrently using HTTP Range requests.

Features

• Parallel Downloading: Utilizes multiple threads to maximize network bandwidth.
• Resumability: Manages download state in a .meta file, allowing users to pause and resume downloads without data loss.
• Thread-Safe Architecture: Employs per-thread file handles and mutex-protected metadata management.
• Graceful Termination: Handles system signals (SIGINT, SIGTERM) to save progress before exiting.

System Requirements

• Compiler: C++17 compatible compiler (GCC, Clang, or MSVC)
• Build System: CMake 3.10+
• Dependencies: libcurl

Installation of Dependencies (macOS)

brew install cmake curl

Installation of Dependencies (Ubuntu/Debian)

sudo apt-get update
sudo apt-get install build-essential cmake libcurl4-openssl-dev

Building the Application

The project uses CMake for build configuration. Follow these steps to compile the source:

1. Create a build directory:

   mkdir build
   cd build

2. Generate build files and compile:

   cmake ..
   make

Upon successful completion, the download_manager executable will be generated in the current directory.

Usage

The application is executed via the command line with the following syntax:

./download_manager <URL> <OutputFilename> [Threads]

Positional Arguments:

1. URL: The direct link to the remote file.
2. OutputFilename: The local path and name for the saved file.
3. Threads (Optional): The number of concurrent connections to spawn. Defaults to 4.

Execution Example:

./download_manager https://www.learningcontainer.com/wp-content/uploads/2020/05/sample-mp4-file.mp4  video.mp4 8

Automated Testing

A test script is provided to demonstrate the "Download -> Interrupt -> Resume" flow. This script automates starting a download, interrupting it mid-way, resuming it, and verifying the logs.

To run the automated test:

chmod +x run_test.sh
./run_test.sh

The results and logs will be displayed in the terminal and recorded in downloader.log.

Graphical User Interface (GUI)

We also provide a Python-based GUI dashboard for easier usage. It allows you to visualize the progress bar and view logs in real-time.

To launch the GUI:

streamlit run app.py

(Note: You must build the C++ project first)

Project Architecture

The application is divided into several logical modules:

• DownloadManager: Orchestrates the download lifecycle, calculates byte ranges (chunks), and manages worker threads.
• ChunkDownloader: The worker unit executed in each thread. It performs HTTP Range requests and manages its local state.
• HttpClient: A wrapper around libcurl for low-level HTTP communication (HEAD and GET requests).
• FileWriter: Handles thread-specific random-access file writes using byte offsets.
• MetadataHandler: Manages persistence of progress. It uses an atomic write/rename strategy to ensure metadata integrity during crashes.

Technical Implementation

• Concurrency: Managed via std::thread and std::atomic flags.
• File I/O: Each thread maintains its own std::fstream handle, seeking to its specific byte offset before writing, which eliminates the need for global file locks.
• Networking: Implements the CURLOPT_RANGE option to request specific byte segments from the server.