scraping millions of records from a public government website
Use Git or checkout with SVN using the web URL.
Work fast with our official CLI. Learn more about the CLI.
Please sign in to use Codespaces.
If nothing happens, download GitHub Desktop and try again.
If nothing happens, download GitHub Desktop and try again.
If nothing happens, download Xcode and try again.
Your codespace will open once ready.
There was a problem preparing your codespace, please try again.
A script and tools to scrape a whole country's population data from a web service and put them in a database.
Threaded to make the ~19 million records faster to download.
Separate script to verify downloaded records and a small Flask app to view the data.
scraping millions of records from a public government website