TPI Production Ready Seeds #131
Conversation
- removed index and validation on ISIN as we had to duplicate some companies that are assessed in multiple sectors; - adds Hong Kong to the geographies list; - updates import script for companies, and override ids to keep the TPI ids;
|
oh I'm also importing Instrument Types, Instruments and LawSectors in this PR, did it directly in the seeds.rb file. |
|
Fixed specs and rubocop and added company_id to MQ and CP assessments download, as this is now used to import the data. |
There was a problem hiding this comment.
Nice! Almost there with TPI data.
Just one thing. CP assessment date is "No date" everywhere.
And the other below could be done here, or later (but then seeds needs to be reimported, I guess they will anyway)
We may also need an extra column to MQ::Assessment, methodology_version to know right away the version, that would be helpful when building question edit feature. This one should not be editable, should come from csv files, so all records in _M1 will have methodolody_version: M1 and so on.
| @@ -46,7 +46,7 @@ class Company < ApplicationRecord | |||
|
|
|||
| validates :ca100, inclusion: {in: [true, false]} | |||
| validates_presence_of :name, :slug, :isin, :market_cap_group | |||
| validates_uniqueness_of :slug, :isin, :name | |||
There was a problem hiding this comment.
right, I missed that thing before, but that wasn't right after having multiple ISINs in one string ;]
There was a problem hiding this comment.
yeah, I missed that too 😅
db/seeds.rb
Outdated
| file = File.open(Rails.root.join('db', 'seeds', 'companies.csv'), 'r') | ||
| CSVImport::Companies.new(file).call | ||
| file = File.open(Rails.root.join('db', 'seeds', 'tpi-companies.csv'), 'r') | ||
| importer = CSVImport::Companies.new(file) |
There was a problem hiding this comment.
Just taking a note here (not a blocker for this PR) to refactor this just a little CSVImport::Companies.new(file, override_id: true)
|
Thanks for the feedback, I'll check it again later today, including the override small refactor. |
simaob
force-pushed
the
feature/production-seeds
branch
from
0e0045b
to
8109cb8
Compare
|
Refactored to have override_id as an option, and added methodology_version, added it to the importing files. The dates are so different that was hard to extract from that... Looking now at the CP Assessments assessment_date |
|
the cp-assessments file had the wrong format on the assessment date... fixed now. Should be good to go, unless there's feedback on my refactor! =D |
| @@ -60,6 +65,12 @@ def csv_converters | |||
|
|
|||
| private | |||
|
|
|||
| def reset_id_seq | |||
| @@ -52,7 +52,8 @@ def assessment_attributes(row) | |||
| publication_date: publication_date(row), | |||
| level: row[:level], | |||
| notes: row[:notes], | |||
| questions: get_questions(row) | |||
| questions: get_questions(row), | |||
| methodology_version: row[:methodology_version] | |||
This PR updates the TPI scripts and data files with the most up to date information, which we can use to seed production.
Main changes are: