Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add scraper for umd buildings #235

Open
javathunderman opened this issue Mar 13, 2022 · 5 comments
Open

add scraper for umd buildings #235

javathunderman opened this issue Mar 13, 2022 · 5 comments

Comments

@javathunderman
Copy link
Contributor

The list of buildings returned from the main buildings endpoint (https://api.umd.io/v1/map/buildings) is very out of date - based on the buildings that do appear on the list, it appears to have been last run between 2017 and 2018.

Endpoint

https://api.umd.io/v1/map/buildings

Additional context

Discovered this bug while working on the new buildings feature for the UMD Discord bot: https://gitlab.com/nahkoots/umd-discord-bot/-/merge_requests/13/

@tybug
Copy link
Contributor

tybug commented Mar 18, 2022

For the record, we don't actually have a scraper for buildings. The current data was manually (?) collected around 2017, as you've surmised.

I'm 95% certain this could be scraped from https://maps.umd.edu/map/index.html, which seems to know the latitude and longitude of every building. So somewhere in its myriad of initial requests it's getting the lat/long data for all umd buildings. I just don't have the time right now to figure out exactly which url they're calling to retrieve this data. If someone else is willing to put the time into this and figure it out, please let me know and I'll likely implement a scraper for this. (eg here's one interesting endpoint it calls, though no lat/long in this one).

Until we can write a scraper for this (which we definitely should), it's probably good enough to manually add buildings for now. We don't have to get 100% coverage of campus, just enough that any glaringly obvious buildings (like iribe) aren't missing.

I've added iribe and pyon-chen. Please let me know of any other major buildings missing and I'll add them.

@javathunderman
Copy link
Contributor Author

I think Yahentamitsi, the Fernandez IDEA Factory, and Johnson-Whittle Hall should be added since they do appear on the UMD Maps webpage. Other than those buildings, I think the API response should be sufficiently up to date/future-proofed. Thanks!

@tybug
Copy link
Contributor

tybug commented Mar 18, 2022

all three added. I'll keep this issue open to track writing a scraper for the buildings.

@tybug tybug changed the title Update buildings API response add scraper for umd buildings Mar 18, 2022
@tybug tybug removed the bug label Aug 31, 2022
@javathunderman
Copy link
Contributor Author

Needs to be updated with building 433 - the School of Public Policy building

@tybug
Copy link
Contributor

tybug commented Sep 27, 2022

updated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants