-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DOP-8208] Fix validation ot includeHeaders #131
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
a496f68
to
40f1496
Compare
40f1496
to
9363298
Compare
9363298
to
6c34e75
Compare
6c34e75
to
79484bb
Compare
Codecov Report
@@ Coverage Diff @@
## develop #131 +/- ##
========================================
Coverage 94.29% 94.30%
========================================
Files 207 207
Lines 7697 7703 +6
Branches 1451 1449 -2
========================================
+ Hits 7258 7264 +6
Misses 320 320
Partials 119 119
|
79484bb
to
2450866
Compare
2450866
to
1bf7c98
Compare
1bf7c98
to
a59d089
Compare
a59d089
to
f4241fe
Compare
f4241fe
to
233bba4
Compare
233bba4
to
6cede4f
Compare
6cede4f
to
1f42431
Compare
1f42431
to
8e1b741
Compare
8e1b741
to
828cfe7
Compare
828cfe7
to
983cfd6
Compare
983cfd6
to
8329d06
Compare
8329d06
to
3fc9807
Compare
maxim-lixakov
approved these changes
Sep 5, 2023
dmitry-pedchenko
approved these changes
Sep 6, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Change Summary
headers
column is written to Kafka with defaultKafka.WriteOptions()
-includeHeaders
had no default value, so instead of raising an exception, column value was just ignored. Now default value is explicitly set toFalse
.Kafka.write_df_to_target
, raise exception if user passedinclude_headers=True
on Spark 2.4, there this option does not exist.DBWriter.run
for Kafka -timestamp
andtimestampType
are not supported by Spark,offset
is not supported by Kafka.Related issue number
Checklist
docs/changelog/next_release/<pull request or issue id>.<change type>.rst
file added describing change(see CONTRIBUTING.rst for details.)