Hea 572/relax data completeness constraint #129
Conversation
Duplicate columns names in wealth_group_df were causing the pd.concat to add the Baseline Wealth Groups.
The BSSs often use labels that match to `price` for `PaymentInKind` or `OtherCashIncome` activities, rather than explicit labels that match `payment_per_time`. Therefore, we correct misidentified labels for these Strategy Types.
Refactor the pipeline so that we can generate and import a fixture for each worksheet (WB, Data, Data2, Data3) within a BSS, instead of needing to consolidate to a single fixture first. We retain the consolidated_fixture and imported_baseline assets and eventually we should disable the standalone imports.
|
This changes the pipeline so that instead of consolidated the all instances (across WB, Data, Data2 and Data3 worksheets) before validating and creating a fixture, it actually validates and creates and imports a fixture for each worksheet separately. The existing So we can load individual worksheets from each BSS for now, and still load the full BSS all in one go later. |
girumb
left a comment
girumb
left a comment
There was a problem hiding this comment.
Minor comments on the doc strings
| other_cash_income_fixture, | ||
| ) -> Output[None]: | ||
| """ | ||
| Imported Django fixtures for a BSS, added to the Django database. |
There was a problem hiding this comment.
Imported 'OtherCashIncome' Django fixtures for a BSS, added to the Django database
| wealth_characteristic_instances, | ||
| ) -> Output[dict]: | ||
| """ | ||
| Valid LivelhoodStrategy and LivelihoodActivity instances from a BSS, ready to be loaded via a Django fixture. |
There was a problem hiding this comment.
Valid LivelihoodStrategy and LivelihoodActivity instances for 'OtherCashIncome' from a BSS, ready to be loaded via a Django fixture.
| wealth_characteristic_instances, | ||
| ) -> Output[dict]: | ||
| """ | ||
| Valid LivelhoodStrategy and LivelihoodActivity instances from a BSS, ready to be loaded via a Django fixture. |
There was a problem hiding this comment.
Valid LivelihoodStrategy and LivelihoodActivity instances for 'WildFoodGathering' from a BSS, ready to be loaded via a Django fixture.
|
@rhunwicks would it be a worthwhile LOE to also enable loading partial sheets, ie, load everything possible without breaking referential integrity? (Possibly with an environment variable to enable these partial loads per environment?) |
No description provided.