Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull Actual Reviews/photos #3

Open
jinwook-k opened this issue May 22, 2020 · 2 comments
Open

Pull Actual Reviews/photos #3

jinwook-k opened this issue May 22, 2020 · 2 comments

Comments

@jinwook-k
Copy link
Owner

There should be a way to pull in actual reviews, ratings, and photos that you actually made (i think)

I don't have a need for it, so i'm not implementing it yet.

@erickhun
Copy link

erickhun commented Sep 27, 2020

That'd be really useful @JinKim7

The workaround I have is to go to the Google Takeout page and export the Google maps data manually . Unfortunately it only contains reviews but not photos :(

Scrapping the public page would actually work better

@jinwook-k
Copy link
Owner Author

If you want to help put that feature in, I can show you what I'm doing to scrape the data.
It's a bit different since google renders the page in their own fancy way.

Normally, you'd think to scrap using some kinds of selectors for the Dom elements, but unfortunately, their rendering cycles makes it where we can't use any Dom selectors.

Instead, we have to scrape one of their network calls for a majority of their static information that gets passed as a huge string blob that's a list of all the reviews, links to images, stats on your profile contribution etc.

  1. What you want to do is open up your network tab in your browser's console and capture the first response that you get (it should be named after your Google Contribution ID, i believe)

  2. on the right-hand side, select the Response tab

  3. To double check it has the information we need, find one of your reviews that has a unique keyword or few keywords -- in my example, i'm using COVID19 --, where i'm pretty sure no other reviews have that kind of pattern of characters; Google pulls 10 default reviews for one of the places your recently visited, so I'm double-checking one of my unique reviews is present in the data I captured, if it is, then all my data should be there.

  4. right-click on the response and Save As... a text file, or you can name it like google reviews response.html or something.

  5. We can then open it up in sublime/text editor and get a wider view of the surrounding text patterns for your own reviews and also finding the link to the images you submitted for each review.

  6. After that, we can then make a new function or class to get these items that we want. I haven't done this yet bc it seems like a different approach/pattern than the profile stats.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants