Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Transfer data to the local server from the bigquery #6

Closed
akorchyn opened this issue Mar 5, 2024 · 1 comment · Fixed by #7
Closed

Transfer data to the local server from the bigquery #6

akorchyn opened this issue Mar 5, 2024 · 1 comment · Fixed by #7
Assignees
@akorchyn

Comments

@akorchyn
Copy link
Member

akorchyn commented Mar 5, 2024 •
edited
Loading

The given work on voting snapshot will require a lot of processing on the data. That might be costly using bigquery api that's why we have to move it first to our local server.
@akorchyn akorchyn self-assigned this Mar 6, 2024
@akorchyn
Copy link
Member Author

akorchyn commented Mar 7, 2024

The data is downloaded to ~/Projects/voting-v2/data/* in gzip CSV format. I will proceed to load it into Postgres.

@akorchyn akorchyn linked a pull request Mar 7, 2024 that will close this issue
Script for loading data from near bigquery csv dump into local postgres with sql tables. #7
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@akorchyn akorchyn

Labels
None yet
Projects
Voting Framework
Status: Done
Milestone
No milestone
1 participant
@akorchyn