Skip to content

Conversation

@hanyuzheng7
Copy link
Contributor

What is the purpose of the change

This is an implementation of SPLIT

Brief change log

Splits a string into an array of substrings based on a delimiter. If the delimiter is not found, then the original string is returned as the only element in the array. If the delimiter is empty, then all characters in the string are split. If either, string or delimiter, are NULL, then a NULL value is returned.
If the delimiter is found at the beginning or end of the string, or there are contiguous delimiters, then an empty space is added to the array.

  • Syntax
    SPLIT(string, delimiter)

  • Arguments
    string: The string need to be split
    delimiter: Splits a string into an array of substrings based on a delimiter

  • Returns
    If the delimiter is not found, then the original string is returned as the only element in the array. If the delimiter is empty, then all characters in the string are split. If either, string or delimiter, are NULL, then a NULL value is returned.

  • Examples

SELECT SPLIT('abcdefg', 'c');
Result: ['ab', 'defg']
  • See also
  1. ksqlDB Split function
    ksqlDB provides a scalar function named SPLIT which splits a string into an array of substrings based on a delimiter.
    Syntax: SPLIT(string, delimiter)
    For example: SPLIT('a,b,c', ',') will return ['a', 'b', 'c'].
    https://docs.ksqldb.io/en/0.8.1-ksqldb/developer-guide/ksqldb-reference/scalar-functions/#split

  2. Apache Hive Split function
    Hive offers a function named split which splits a string around a specified delimiter and returns an array of strings.
    Syntax: array split(string str, string pat)
    For example: split('a,b,c', ',') will return ["a", "b", "c"].
    https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF

  3. Spark SQL Split function
    Spark SQL also offers a function named split, similar to the one in Hive.
    Syntax: split(str, pattern[, limit])
    Here, limit is an optional parameter to specify the maximum length of the returned array.
    For example: split('oneAtwoBthreeC', '[ABC]', 2) will return ["one", "twoBthreeC"].
    https://spark.apache.org/docs/latest/api/sql/index.html#split

  4. Presto Split function
    Presto offers a function named split which splits a string around a regular expression and returns an array of strings.
    Syntax: array split(string str, string regex)
    For example: split('a.b.c', '.') will return ["a", "b", "c"].
    https://prestodb.io/docs/current/functions/string.html

Verifying this change

This change added tests in CollectionFunctionsITCase.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (yes / no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
  • The serializers: (yes / no / don't know)
  • The runtime per-record code paths (performance sensitive): (yes / no / don't know)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
  • The S3 file system connector: (yes / no / don't know)

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

[BP-1.17][FLINK-32706][table] Add built-in SPLIT_STRING function
@flinkbot
Copy link
Collaborator

flinkbot commented Aug 9, 2023

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@MartijnVisser
Copy link
Contributor

Closing this PR since we don't backport new features into existing releases

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants