Skip to content

[FEATURE] Add ARRAY_TO_CSV helper function #4587

@Trikenstein

Description

@Trikenstein

Is your feature request related to a problem?
Using PPL 3.4.0, it would be very convenient to have an array helper function to flatten an array content into a CSV string.

As of 2025-10-16, the PPL Collection Functions has a REDUCE function which can fulfill this task. However it is rather clumsy.

Example:

source=security-auditlog-2024.05
| eval myArray=array('GET', 'READ', 'WRITE')
| eval myArray_CSV = reduce(myArray, '', (acc, x) -> CONCAT_WS(',', acc, x))
| fields myArray, myArray_CSV
| head 1

The output looks like (notice the , prefix in the CSV result ,GET,READ,WRITE)

{
  "schema": [
    {
      "name": "myArray",
      "type": "array"
    },
    {
      "name": "myArray_CSV",
      "type": "string"
    }
  ],
  "datarows": [
    [
      [
        "GET",
        "READ",
        "WRITE"
      ],
      ",GET,READ,WRITE"
    ]
  ],
  "total": 1,
  "size": 1
}

What solution would you like?
It would be more convenient to have a utility function named ARRAY_TO_CSV to build the CSV string. The above query would have a more user-friendly syntax.

source=security-auditlog-2024.05
| eval myArray=array('GET', 'READ', 'WRITE')
| eval myArray_CSV = ARRAY_TO_CSV(myArray)
| fields myArray, myArray_CSV
| head 1

Furthermore, the ARRAY_TO_CSV should accept and optional 2nd parameter which is the delimiter string. This would allow a better formatting of the CSV string. The default value as , (comma, no space)

// CSV with comma-space as separator
ARRAY_TO_CSV(myArray, ", ")  -> "GET, READ, WRITE"

// CSV with space-pipe-space as separator
ARRAY_TO_CSV(myArray, " | ")  -> "GET | READ | WRITE"

What alternatives have you considered?
Use the REDUCE function, which has 2 inconveniences:

  • complicate syntax
  • resulting CSV always has an extra separator string, either at the beginning or the end.

Do you have any additional context?
The ARRAY_TO_CSV function if available, would help to improve the output of the collector operators TAKE, LIST, VALUES of the stats command.

Metadata

Metadata

Assignees

No one assigned

    Labels

    PPLPiped processing languagecalcitecalcite migration releatedfeature

    Type

    No type

    Projects

    Status

    Not Started

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions