Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
In past years we've turned off all the scrapers until we could quickly verify that things are looking normal on state sites. We've seen enough weirdness where we were getting 2017 bills despite hitting the API with the 2016 slug or vice versa that this sort of became unofficial protocol.
I know a few states are in special session & a few have prefiles, but we can turn those back on really quick, or potentially even vet them before I make the big push to task-definitions to turn them off.
When turning back on we also want to give each state a quick once-over to make sure that there weren't any hard-coded year/session values (I think we got all of these, but they creep in from time to time)
I was thinking I'd flip this switch to turn things off next week, and we can use this ticket as a master list to get things turned back on after vetting.
How to help: If you've run a states bill/votes scraper and verified it is getting 2019 data (or is in a special session) and taken a look at the code to look for hard-coded session/year values then leave a comment and we'll update the list below. Next week I'll make a PR to turn off all non-checked states and then as states are checked we'll turn them back on. (If a state needs work first, we'll discuss that in a separate ticket, like #2554)
Sounds good to me. We've got a few states already running the 2019 prefiles that look fine; i've checked these off above.
I've also got a patch incoming with a few states that have added empty 2019 sessions, so I added commented out session entries and added 'em to the ignore list, just to avoid bobsled errors. Sounds like i can hold off on doing any more of those if we're going to turn them off, so i'll push that commit later today.