Skip to content
This repository has been archived by the owner on Oct 12, 2021. It is now read-only.

Scraper fails #36

Open
svennickel opened this issue Apr 22, 2021 · 2 comments
Open

Scraper fails #36

svennickel opened this issue Apr 22, 2021 · 2 comments

Comments

@svennickel
Copy link

I try to run the simple demo script:

from app_store_scraper import AppStore
from pprint import pprint

minecraft = AppStore(country="nz", app_name="minecraft")
minecraft.review(how_many=20)

pprint(minecraft.reviews)
pprint(minecraft.reviews_count)

It fails as follows:

2021-04-22 12:09:55,491 [INFO] Base - Searching for app id
Traceback (most recent call last):
  File "C:\Users\snick\Documents\Entwicklung\Python\App-Rezensionen\scrape_test_ios.py", line 4, in <module>
    minecraft = AppStore(country="nz", app_name="minecraft")
  File "C:\Users\snick\AppData\Local\Programs\Python\Python39\lib\site-packages\app_store_scraper\app_store.py", line 23, in __init__
    super().__init__(
  File "C:\Users\snick\AppData\Local\Programs\Python\Python39\lib\site-packages\app_store_scraper\base.py", line 47, in __init__
    app_id = self.search_id()
  File "C:\Users\snick\AppData\Local\Programs\Python\Python39\lib\site-packages\app_store_scraper\base.py", line 175, in search_id
    app_id = re.search(pattern, self._response.text).group(1)
AttributeError: 'NoneType' object has no attribute 'group'
@flexbox
Copy link

flexbox commented Jun 8, 2021

I got the same issue on macOS

I run my script with

python3 app.py
2021-06-08 11:40:15,926 [INFO] Base - Searching for app id
Traceback (most recent call last):
  File "/Users/cerebra/workspace/stablehouse/scraping-app-store/app.py", line 13, in <module>
    minecraft = AppStore(country="nz", app_name="minecraft")
  File "/opt/homebrew/lib/python3.9/site-packages/app_store_scraper/app_store.py", line 23, in __init__
    super().__init__(
  File "/opt/homebrew/lib/python3.9/site-packages/app_store_scraper/base.py", line 47, in __init__
    app_id = self.search_id()
  File "/opt/homebrew/lib/python3.9/site-packages/app_store_scraper/base.py", line 175, in search_id
    app_id = re.search(pattern, self._response.text).group(1)
AttributeError: 'NoneType' object has no attribute 'group'

@svennickel

what's your output of

python3 --version
Python 3.9.5

@Lexstok
Copy link

Lexstok commented Jul 12, 2021

The very first time you visit Google nowadays (at least in my country) you need to give consent for them to store cookies and track you (or not). In the code current to this date there is no cookie included and so the search in base.py blocks because it does not receive the result page it expects, but instead receives the content page.
This can probably be solved by adding a cookie to the request indicating that you already visited google...

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants