Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Update IL scraper for older sessions #105
I see that the ILGA has made some changes to their URL structure since last I worked on this code. I poked around and worked out how to get URLs for the four previous sessions of the assembly. I haven't exhaustively tested things, but I did run the entire bill scraper and several days later, there were no fatal errors and the JSON I've looked at seems sound.
It does seem that there are six kinds of documents which are not currently being passed to bill.add_document and I'll see if I can find time to check those out and process them, but I figured I'd offer what works here rather than defer that indefinitely...