Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: Add snapshotting started message to ingestion process #1377

Merged
merged 7 commits into from
Apr 4, 2023

Conversation

karolisg
Copy link
Contributor

@karolisg karolisg commented Apr 4, 2023

No description provided.

@karolisg karolisg marked this pull request as ready for review April 4, 2023 08:33
@coveralls
Copy link

coveralls commented Apr 4, 2023

Pull Request Test Coverage Report for Build 4606530520

  • 38 of 50 (76.0%) changed or added relevant lines in 6 files are covered.
  • 2115 unchanged lines in 39 files lost coverage.
  • Overall coverage decreased (-1.9%) to 72.054%

Changes Missing Coverage Covered Lines Changed/Added Lines %
dozer-orchestrator/src/pipeline/connector_source.rs 1 2 50.0%
dozer-ingestion/src/connectors/object_store/tests/local_storage_tests.rs 9 20 45.0%
Files with Coverage Reduction New Missed Lines %
dozer-orchestrator/src/simple/orchestrator.rs 1 58.21%
dozer-api/src/generator/protoc/generator/implementation.rs 4 86.67%
dozer-api/src/rest/api_generator.rs 5 71.0%
dozer-cache/src/cache/index/mod.rs 5 94.05%
dozer-core/src/executor/source_node.rs 5 94.74%
dozer-ingestion/src/connectors/object_store/table_reader.rs 5 89.09%
dozer-core/src/epoch.rs 6 73.89%
dozer-orchestrator/src/pipeline/connector_source.rs 6 86.36%
dozer-sql/src/pipeline/expression/python_udf.rs 7 66.67%
dozer-api/src/cache_builder/mod.rs 8 59.09%
Totals Coverage Status
Change from base Build 4605596598: -1.9%
Covered Lines: 33438
Relevant Lines: 46407

💛 - Coveralls

@snork-alt snork-alt merged commit c0bf3ca into getdozer:main Apr 4, 2023
@chubei
Copy link
Contributor

chubei commented Apr 4, 2023

@snork-alt Is this intended to be merged? There's still todo! in code.

@karolisg
Copy link
Contributor Author

karolisg commented Apr 4, 2023

@snork-alt Is this intended to be merged? There's still todo! in code.

This PR is just for sending snapshotting started/done ingestion messages. I think logic of how it will be used should be implemented in a separate PR.

@chubei
Copy link
Contributor

chubei commented Apr 4, 2023

I was worried the todo! would abort the dozer process.

@karolisg
Copy link
Contributor Author

karolisg commented Apr 4, 2023

I was worried the todo! would abort the dozer process.

Got it. Anyway, I have changed it to simple comment https://github.com/getdozer/dozer/pull/1377/files#diff-dfd9b1553d77e73ce70cadd635b4f5e6a925b5c246d31a740975ff2d3e27ad30R214-R215

@chubei
Copy link
Contributor

chubei commented Apr 4, 2023

Oh I was looking at a outdated version of the PR. Sorry.

@chubei chubei deleted the feature/snapshotting-started branch May 22, 2023 14:12
@chubei chubei mentioned this pull request May 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants