Skip to content

Google Image Search API Caps at 100 Images Download #7

@DragonflyRobotics

Description

@DragonflyRobotics

Description
The Google Image Search API was updated to have a limit of 100 images per query. This means that consecutively downloading, for example, 200 images will fail.

To Reproduce
To reproduce the behavior, run the following code:

from MAGIST.Utils.WebScraper.google import GoogleScraper

g = GoogleScraper("config.json")
g.download_raw_img_dataset("cars", 200, "Data/")

Expected behavior
It should ideally download 200 images and verify their integrity before downloading.

Screenshots
image
image

Additional context
I have teamed up with @arrrlo who made the GoogleImageSearch module for Python with the issue also posted on @arrrlo's repo.

This requires the module pyparsing==2.4.2 which is part of a circular dependency issue(#4 ).

Metadata

Metadata

Labels

bugSomething isn't working

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions