Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][Connectors-V2][File]support assign encoding for file source/sink #6489

Merged
merged 2 commits into from Mar 14, 2024

Conversation

ShaunWuu
Copy link
Contributor

Purpose of this pull request

This pull request support assign encoding for file source/sink

Does this PR introduce any user-facing change?

Yes.
A config to assign encoding for file source/sink. The name of the config is "encoding".
This pr is within the unreleased branch dev.

One of the example:

LocalFile {
    path = "/tmp/hive/warehouse/test2"
    file_format_type = "text"
    encoding = "gbk"
}

How was this patch tested?

Unit tests are as the followings:

  • org.apache.seatunnel.connectors.seatunnel.file.writer.ReadStrategyEncodingTest

E2E test cases are as the followings:

  • org.apache.seatunnel.e2e.connector.file.local.LocalFileIT

Check list

@hailin0
Copy link
Contributor

hailin0 commented Mar 12, 2024

cc @TyrantLucifer

@ShaunWuu
Copy link
Contributor Author

force-pushed the dev-240304-fileEncoding branch from d5adc8c to 424ca42

This force push deleted a redundant line of code.

@lightzhao
Copy link
Contributor

lightzhao commented Mar 12, 2024

#5973 cc @EricJoy2048

Copy link
Member

@Hisoka-X Hisoka-X left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM

@hailin0 hailin0 merged commit d159fbe into apache:dev Mar 14, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants