Skip to content

Comments

[INLONG-5703][Manager] Add separator-related fields for some sources#5706

Merged
dockerzhang merged 8 commits intoapache:masterfrom
e-mhui:INLONG-5703
Aug 26, 2022
Merged

[INLONG-5703][Manager] Add separator-related fields for some sources#5706
dockerzhang merged 8 commits intoapache:masterfrom
e-mhui:INLONG-5703

Conversation

@e-mhui
Copy link
Contributor

@e-mhui e-mhui commented Aug 25, 2022

Prepare a Pull Request

[INLONG-5703][Manager] Get separator from enum 'DataSeparator' and remove fieldDelimiter from properties of PulsarSource

Motivation

  • DataSeparator is an ascii code, so we should get separator from enum DataSeparator
  • if not remove fieldDelimiter, it will be parse into sql option. for example:
"properties": {
  "properties.auth-params": "token:xxxx",
  "properties.auth-plugin-classname": "org.apache.pulsar.client.impl.auth.AuthenticationToken",
  "fieldDelimiter": "|"
}

@healchow healchow changed the title [INLONG-5703][Manager] Get separator from enum 'DataSeparator' and remove fieldDelimiter from properties of PulsarSource [INLONG-5703][Manager] Add separator-related fields for some sources Aug 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature][Manager] Add separator-related fields for File, Kafka, and Pulsar sources

3 participants