Skip to content

Add Hexia platform parser#60

Merged
wtfloris merged 4 commits into
wtfloris:masterfrom
ventilaar:hexia
Nov 24, 2024
Merged

Add Hexia platform parser#60
wtfloris merged 4 commits into
wtfloris:masterfrom
ventilaar:hexia

Conversation

@ventilaar
Copy link
Copy Markdown
Contributor

@ventilaar ventilaar commented Nov 12, 2024

As mentioned in #53. Wooniezie and many other websites just buy the platform from a common third party.

Endpoint to get the data, no data needs to be posted:
https://{corp}api.hexia.io/api/v1/actueel-aanbod?limit=99&sort=-publicationDate

Where corp is the slug of the corporation which you want to parse. You can get this by simply filtering the request path in inspect element in your browser. The name is always appended with api and is sent to hexia.io, though sometimes it's also zig365.nl, but using hexia.io seems to work anyway, so no special handling is needed. Current supported targets:

  • wooniezie
  • hwwonen
  • thuisindeachterhoek
  • dewoningzoeker
  • thuispoortstudenten
  • thuispoort
  • zuidwestwonen
  • thuisinlimburg
  • klikvoorwonen
  • mercatus-aanbod
  • woninginzicht
  • mosaic-plaza
  • oostwestwonen
  • noordveluwe
  • woninghuren
  • woonnethaaglanden
  • hollandrijnland
  • antares
  • studentenenschede
  • thuiskompas
  • woonkeusstedendriehoek
  • woontij
  • frieslandhuurt
  • svnk

To generate a direct link to the listing we need to map the corp to the full website path of the corporation which is different and not reported in the API. So for this I created a dictionary with mappings from the corp slug to the full listing URL, you can get this by simply opening a random listing in your browser.

To install this in the system it's the same as with the DAK endpoints, but instead of regions it's the corporate slugs.

@ventilaar ventilaar marked this pull request as ready for review November 12, 2024 11:02
@wtfloris
Copy link
Copy Markdown
Owner

wtfloris commented Nov 12, 2024

This is a great find, thank you (and for your other contributions as well)!

Gonna put a few of these in test to see how it goes, I wonder if we're going to run into any rate limiting here.

One small thing that is missing is that address = should be home.address =.

  • Homes are written to database, filling address, city, price and url
  • House number additions are handled properly
  • URLs lead to the correct page
  • Prices are rounded integers
  • Scraper either gets all homes or sorts by "Newest first"

@wtfloris wtfloris mentioned this pull request Nov 12, 2024
19 tasks
@wtfloris
Copy link
Copy Markdown
Owner

Ran into some issues with some of the websites: looks like not all results have all required keys and Mercatus gives me a 503.

Others look fine so I'll add most to Hestia soon.

@wtfloris wtfloris merged commit 0651672 into wtfloris:master Nov 24, 2024
@ventilaar ventilaar deleted the hexia branch November 24, 2024 11:15
xenbyte pushed a commit to xenbyte/hermes that referenced this pull request Mar 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants