2018-2019 school import #38432
Merged
2018-2019 school import #38432
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
8 tasks
clareconstantine
approved these changes
Jan 11, 2021
| else | ||
| School.seed_from_s3 | ||
| #else | ||
| #School.seed_from_s3 |
There was a problem hiding this comment.
just confirming - this is the change to prevent the seeding from happening with the new file before we do a dry run?
There was a problem hiding this comment.
Yeah pretty much -- with school districts I actually did a non-dry run on staging (which appeared to have the entire school district dataset, which test didn't surprisingly), then a non-dry run on production. Basically just wanted to be able to run it manually to keep an eye on it if anything went wrong.
| id: row['NCESSCH'].to_i.to_s, | ||
| name: row['SCH_NAME'].upcase, | ||
| # Four schools with addresses longer than 50 characters (DB column limit) | ||
| # Also four schools with second address line longer than 30 characters. |
There was a problem hiding this comment.
What happens with those schools?
There was a problem hiding this comment.
I'm truncating them (was getting DB errors on import without the truncation, figured cutting these four schools down in length wasn't a huge deal and a lot easier than expanding the allowed character limit for the columns via migration).
| new_attributes: new_attributes, | ||
| &parse_row | ||
| ) | ||
| ensure |
|
Production import: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Updates to import 2018-2019 schools. This PR includes:
state_school_id, which (without these changes) is blocked by the foreign key relationship tostate_cs_offerings. The approach here is to destroy (delete) the existing state CS offerings temporarily, update the school'sstate_school_id, then recreate thestate_cs_offerings.To do (specific to this import):
(other) to do in NCES import space:
school_stats_by_year.Testing story
I've included tests to cover the dry run import of a CSV, and the new reconstitution of deleted state CS offerings.
I also tested the import manually, by seeding my schools table from scratch (without the new block for 2018-2019 data), then after adding the block running `School.seed_from_s3(stub_school_data: false)
Result:
Reviewer Checklist: