Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Google Image Search API Caps at 100 Images Download #7

Open
DragonflyRobotics opened this issue Jun 20, 2022 · 1 comment
Open

Google Image Search API Caps at 100 Images Download #7

DragonflyRobotics opened this issue Jun 20, 2022 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@DragonflyRobotics
Copy link
Member

DragonflyRobotics commented Jun 20, 2022

Description
The Google Image Search API was updated to have a limit of 100 images per query. This means that consecutively downloading, for example, 200 images will fail.

To Reproduce
To reproduce the behavior, run the following code:

from MAGIST.Utils.WebScraper.google import GoogleScraper

g = GoogleScraper("config.json")
g.download_raw_img_dataset("cars", 200, "Data/")

Expected behavior
It should ideally download 200 images and verify their integrity before downloading.

Screenshots
image
image

Additional context
I have teamed up with @arrrlo who made the GoogleImageSearch module for Python with the issue also posted on @arrrlo's repo.

This requires the module pyparsing==2.4.2 which is part of a circular dependency issue(#4 ).

@DragonflyRobotics DragonflyRobotics added the bug Something isn't working label Jun 20, 2022
@DragonflyRobotics DragonflyRobotics self-assigned this Jun 20, 2022
@DragonflyRobotics DragonflyRobotics moved this from To do to In progress in MAGIST v0.1.0 AlphaBird Jun 22, 2022
@DragonflyRobotics DragonflyRobotics removed this from In progress in MAGIST v0.1.0 AlphaBird Jun 23, 2022
@DragonflyRobotics DragonflyRobotics removed this from the MAGIST v0.1.0 AlphaBird milestone Jun 26, 2022
@DragonflyRobotics
Copy link
Member Author

Removed from milestone as there is no current solution available. I will keep the issue open though as @arrrlo and I come up with a work-around.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
No open projects
Development

No branches or pull requests

1 participant