Skip to content

Conversation

@argush3
Copy link
Collaborator

@argush3 argush3 commented Oct 29, 2025

Issue #: /bcgov/entity#31025

Description of changes:

  • Add colin extract view ddl which contains db views used for data analysis work
  • Misc updates to core colin extract ddl + tracking table backup/restore shell scripts

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the lear license (Apache 2.0).

@argush3 argush3 self-assigned this Oct 29, 2025
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an optional ddl for the colin extract that contains views used for data analysis. If you are doing analysis on eligible businesses for migration and bad address data analysis, you will need these views.

It needs to be run after the transfer script is run to load data into the core colin extract schema.

references mig_batch,
corp_num varchar(10) not null
corp_num varchar(10) not null,
notes varchar(600)
Copy link
Collaborator Author

@argush3 argush3 Oct 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new notes field that is useful if we need to note anything specific about a corp that we are migrating in the batches of corps. for now we have used it for backfill test dataset to indicate which filings or scenarios we are using a given corp for

Comment on lines +984 to +987
CREATE TABLE IF NOT EXISTS email_domain_groups (
email_domain VARCHAR(255) PRIMARY KEY,
group_name VARCHAR(100) NOT NULL
);
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data analysis helper table that's not required by core corps migration

@argush3 argush3 force-pushed the 31025_data_analysis_related_colin_extract_updates branch from 9e21d6d to 36ac3c5 Compare October 29, 2025 18:47
ALTER MATERIALIZED VIEW mv_addr_quality_by_corp
owner to postgres;

CREATE MATERIALIZED VIEW mv_legacy_corps_data AS
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

main view used to quickly analyze if businesses are eligible for migration or how many businesses match common criteria of interest

@argush3 argush3 marked this pull request as ready for review October 29, 2025 18:51
Copy link
Collaborator

@JazzarKarim JazzarKarim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!! 👍

@argush3 argush3 merged commit d6ea0dc into bcgov:main Oct 29, 2025
1 check passed
@argush3 argush3 deleted the 31025_data_analysis_related_colin_extract_updates branch October 29, 2025 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants