Skip to content

feat(query): Array Functions Supporting ARRAY and VARIANT Types #18213

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 23, 2025

Conversation

b41sh
Copy link
Member

@b41sh b41sh commented Jun 23, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

This PR introduces a set of array functions to Databend, designed to enhance compatibility with Snowflake and provide more flexible array manipulation capabilities. These functions are implemented to seamlessly support both native ARRAY types and VARIANT types containing arrays.

Functions:

The following functions originally supported the Array type but not the Variant type:

  • ARRAY_APPEND(array, element): Appends an element to the end of an array.
  • ARRAY_CONTAINS(array, element): Returns true if the array contains the element.
  • ARRAY_FLATTEN(array): Flattens a nested array into a single-dimensional array.
  • ARRAY_INDEXOF(array, element): Returns the index of the first occurrence of an element in an array.
  • ARRAY_PREPEND(element, array): Prepends an element to the beginning of an array.
  • ARRAY_REMOVE_FIRST(array): Removes the first occurrence of an element from an array.
  • ARRAY_REMOVE_LAST(array): Removes the last occurrence of an element from an array.
  • ARRAY_SLICE(array, start, end): Extract a sub array using slice between start and end argument.
  • ARRAY_UNIQUE(array): Returns the number of unqiue elements in the array.

The following functions originally supported the Variant type but not the Array type:

  • ARRAY_EXCEPT(array1, array2): Returns a new ARRAY that contains the elements from array1 and not in array2.
  • ARRAY_OVERLAP(array1, array2): Compares whether two arrays have at least one element in common.

The following functions have been added:

  • ARRAY_COMPACT(array): Removes all NULL values from an array.
  • ARRAY_REMOVE(array, element): Removes all occurrences of an element from an array.
  • ARRAY_REVERSE(array): Reverses the order of elements in an array.

Key Features:

  • Snowflake Compatibility: These functions are named and behave consistently with their counterparts in Snowflake, simplifying migrations and cross-platform development.
  • ARRAY and VARIANT Support: The functions are designed to work seamlessly with both native ARRAY types and VARIANT types that contain array data. This provides flexibility when dealing with semi-structured data.
  • Comprehensive Array Manipulation: The added functions cover a wide range of common array manipulation tasks, making it easier to process and transform array data within Databend.

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Jun 23, 2025
@b41sh b41sh force-pushed the feat-variant-array-func branch from d3c85d7 to f5326aa Compare June 23, 2025 11:52
Copy link
Contributor

🤖 Smart Auto-retry Analysis (Annotations-based)

The workflow run 15823472485 failed and has been analyzed for retryable errors using job annotations.

Analysis Results:

  • Total failed/cancelled jobs: 3
  • Jobs with retryable errors: 2
  • Jobs with code/test issues: 1
  • 2 job(s) have been automatically retried due to infrastructure issues detected in annotations (runner communication, network timeouts, resource exhaustion, etc.)

You can monitor the retry progress in the Actions tab.

Job Analysis (based on annotations):

  • linux / build (x86_64, X64): 🔄 Retryable (infrastructure) - Infrastructure issue detected (1 annotations)
  • linux / test_unit: 🔄 Retryable (infrastructure) - Infrastructure issue detected (1 annotations)
  • ready: ❌ Not retryable (code/test) - Code/test issue detected (1 annotations)

This is an automated analysis and retry triggered by the smart retry workflow using job annotations.

@b41sh b41sh requested a review from sundy-li June 23, 2025 13:28
@b41sh b41sh marked this pull request as ready for review June 23, 2025 13:28
@b41sh b41sh merged commit 588d3c1 into databendlabs:main Jun 23, 2025
175 of 180 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants