Flickr Scrapper Ultra

Based on flickr_scrapper

This script fetches photos from Flickr based on specified tags and saves the photo information and images to a local directory.

Features

• Fetch photos from Flickr using specified tags.
• Save photo information in JSON format.
• Download and save images in the specified format and size.
• Parallel processing for faster downloads.

Requirements

• Python 3.x
• Required Python packages (listed in requirements.txt):
  • requests
  • tqdm

Installation

1. Clone the repository:

   git clone https://github.com/arcanite24/flickr_scraper_ultra.git
   cd flickr_scraper_ultra

2. Install the required packages:

   pip install -r requirements.txt

3. Obtain a Flickr API key and save it in a file named FLICKR_API_KEY in the root directory of the project.

Usage

Run the script with the following command:

python main.py <tags> [--output OUTPUT] [--cores CORES] [--per_page PER_PAGE] [--sort SORT] [--max_pages MAX_PAGES] [--size SIZE] [--format FORMAT] [--no_download]

Arguments

• tags (required): Tags to search for photos.
• --output (optional): Output folder for saving photo information (default: output).
• --cores (optional): Number of cores to use for parallel processing (default: 16, -1 to use all available cores).
• --per_page (optional): Number of photos per page (min 5, max 500, default: 500).
• --sort (optional): Sort order of the photos (default: relevance).
• --max_pages (optional): Maximum number of pages to fetch (default: 10, -1 to fetch all available pages).
• --size (optional): Size suffix for the images (e.g., s, q, t, m, n, w, z, c, b, h, k, 3k, 4k, f, 5k, 6k, o, default: b).
• --format (optional): Format of the images (e.g., jpg, png, default: png).
• --no_download (optional): Fetch the session data without downloading the images.

Example

python main.py "nature,landscape" --output my_photos --cores 8 --per_page 100 --sort interestingness-desc --max_pages 5 --size m --format jpg --no_download

This command will fetch photos tagged with "nature" and "landscape", save the information and images in the my_photos directory, use 8 cores for parallel processing, fetch 100 photos per page, sort by interestingness in descending order, fetch up to 5 pages, save images in medium size and JPG format, and skip downloading the images.

License

This project is licensed under the MIT License.