Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Image scraper button from MLS URL doesn't seem to work in production deploy. 502 bad gateway... BUT WORKS a little later! not reliable... #15

Open
stanjdev opened this issue Oct 19, 2020 · 1 comment
Open

Image scraper button from MLS URL doesn't seem to work in production deploy. 502 bad gateway... BUT WORKS a little later! not reliable... #15

stanjdev opened this issue Oct 19, 2020 · 1 comment
Assignees
@stanjdev
Projects
Realyzer progress

Comments

@stanjdev
Copy link
Owner

stanjdev commented Oct 19, 2020 •
edited

address. search it on redfin, auto-populate photo, details. also a map. map on top of page. itemized breakdown two columns.

try: https://davidwalsh.name/run-scraping-api-phearjs
@stanjdev stanjdev self-assigned this Oct 19, 2020
@stanjdev stanjdev added this to To do in Realyzer progress via automation Oct 19, 2020
@stanjdev stanjdev changed the title Scrape details from MLS redfin, etc. Scraping - scrape details from MLS redfin, etc. Oct 19, 2020
@stanjdev stanjdev moved this from To do to In progress in Realyzer progress Oct 20, 2020
@stanjdev stanjdev changed the title Scraping - scrape details from MLS redfin, etc. Scraping - scrape details from MLS redfin. photos, maps Oct 20, 2020
@stanjdev
Copy link
Owner Author

stanjdev commented Oct 21, 2020 •
edited

I can scrape basic simple images, stronger built websites like realtor.com will crash my scraper and entire site, zillow.com must be leading to a bot page verification site, but redfin works and other sites too.

Only problem is, every website is different. need logic to limit how many photos I want to display. cuz it's ANY photo. little icons, to house photos, social media icons. etc.

  • Loading screen thing from Kelp for image scraper instead of button? When user pastes in url? but can cause too many rerenders..

@stanjdev stanjdev changed the title Scraping - scrape details from MLS redfin. photos, maps Loading screen thing from Kelp for image scraper instead of button? so you at least know it's working Nov 3, 2020
@stanjdev stanjdev changed the title Loading screen thing from Kelp for image scraper instead of button? so you at least know it's working Loading screen thing from Kelp for image scraper instead of button? When user pastes in url? but can cause too many rerenders.. Nov 3, 2020
@stanjdev stanjdev changed the title Loading screen thing from Kelp for image scraper instead of button? When user pastes in url? but can cause too many rerenders.. Image scraper button from MLS URL doesn't seem to work in production deploy. 502 bad gateway... Nov 3, 2020
@stanjdev stanjdev changed the title Image scraper button from MLS URL doesn't seem to work in production deploy. 502 bad gateway... Image scraper button from MLS URL doesn't seem to work in production deploy. 502 bad gateway... BUT WORKS a little later! not reliable... Nov 3, 2020
@stanjdev stanjdev moved this from In progress to To do in Realyzer progress Nov 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@stanjdev stanjdev

Labels
None yet
Projects
Realyzer progress
  
To do
Milestone
No milestone
Development

No branches or pull requests
1 participant
@stanjdev