This is an experiment to scrape the Transports Quebec infrastructure database and save the data in different, easily usable formats.
Requirements (Tested On)
Not tested with newer versions of the above. YMMV.
I currently have not plan to "support" this project. However, if you find and fix issues (e.g. stuff that does not work anymore because the HTML being scraped has been changed) or add features, feel free to send me pull requests.
If you find an issue that yourself have no plan to fix, feel free to open a ticket to let me know. Maybe by that time I will have found a portal to another dimension where I have extra time or a clone that would allow me to work on it.
I had not looked at this code in a long time and I was curious to see if it still worked. It did not.
However, by doing some tests in the Scrapy shell and checking HTML source code, I realized little would be necessary to fix things. So I found some time to update the scraper to have it work on the current MTQ website. Mostly, I had to change the base URL, the table IDs and XPath selector to get the structure photo URL.
NOTE: I did not test the code with the latest and greatest Scrapy version. Instead, to save myself trouble, I went with one of the oldest available version on PyPI (0.14.4) which did not require any change in my code.