Skip to content
Downloading Images with multiprocessing using python requests and multiprocessing.
Python
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.gitignore
LICENSE
README.md
cats.txt
image_downloader.py

README.md

image_downloader_multiprocessing_python

Here we will use multiprocessing to download images in batch with python.

This saved me a lot of time while downloading images.

Installation

Clone the repository to your machine.

git clone https://github.com/nOOBIE-nOOBIE/image_downloader_multiprocessing_python

Install the requirements

pip install -r requirements.txt

Usage

python3 image_downloader.py <filename_with_urls_seperated_by_newline.txt> <num_of_process>

This will read all the urls in the text file and download them into a folder with name same as the filename. num_of_process is optional.(by default it uses 10 process)

Example

python3 image_downloader.py cats.txt

cat images downloading

cat image downloading

Benchmark

1183 images in 121.99 seconds with 10 process.

Images folder sample

cats

You can’t perform that action at this time.