Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

320: add version to _dlt_loads table #466

Merged
merged 7 commits into from
Jul 28, 2023

Conversation

codingcyclist
Copy link
Collaborator

Resolves 320

@netlify
Copy link

netlify bot commented Jul 1, 2023

Deploy Preview for dlt-hub-docs canceled.

Name Link
🔨 Latest commit ba94e54
🔍 Latest deploy log https://app.netlify.com/sites/dlt-hub-docs/deploys/64c374b0fbc0390008962da2

@codingcyclist codingcyclist changed the title 320: add load_id to _dlt_versions table 320: add version to _dlt_loadss table Jul 1, 2023
@codingcyclist codingcyclist changed the title 320: add version to _dlt_loadss table 320: add version to _dlt_loads table Jul 1, 2023
Copy link
Collaborator

@rudolfix rudolfix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code changes are solid and all is ok there! thanks!
we still need to improve some tests. I enabled CI for your fork and a few tests are not passing

  1. small linter issue (if you have poetry and make you can just do make dev && make lint)
  2. common test problems: https://github.com/dlt-hub/dlt/actions/runs/5430315589/jobs/9969881019?pr=466

why the common tests are failing: each schema has a content hash used to detect schema changes. increasing the version changed the hash (when I load fixtures in tests, your migration modifies the content and we get a different hash - then tests that check hashes fail)
to fix: we need a v6 version of this fixture: ethereum_schema_v5.yml. I generate it with a dlt command:

dlt schema tests/common/cases/schemas/eth/ethereum_schema_v5.yml --remove-defaults

and save the output to new name. from it you can get the new content hash and put it into IMPORTED_VERSION_HASH_ETH_V5 constant (used by the tests)

btw. we'll upgrade our CI to be more friendly for PRs from forks: #476

if the above looks too complicated or unclear just ping us, someone will take over the final fixes

@codingcyclist
Copy link
Collaborator Author

Hi @rudolfix Thanks a lot for your comments :) I took a stab at them and things look good locally. However, the nefly CI seems to be blocking the test from running on my fork. Is there anything I can do about that 🤔 ?

@rudolfix
Copy link
Collaborator

@codingcyclist I just unblocked CI. again... maybe it blocks PRs after some inactivity. will check out

Copy link
Collaborator

@rudolfix rudolfix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@rudolfix rudolfix merged commit 52c0c11 into dlt-hub:devel Jul 28, 2023
20 of 36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add load_id to _dlt_versions table
2 participants