Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement concat_elements_dyn kernel #1755

Closed
Tracked by #1772
alamb opened this issue May 26, 2022 · 4 comments · Fixed by #3763
Closed
Tracked by #1772

Implement concat_elements_dyn kernel #1755

alamb opened this issue May 26, 2022 · 4 comments · Fixed by #3763
Assignees
Labels
arrow Changes to the arrow crate enhancement Any new improvement worthy of a entry in the changelog good first issue Good for newcomers

Comments

@alamb
Copy link
Contributor

alamb commented May 26, 2022

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
The idea is to add kernels to support concatenating arrays element wise, such as concat_elements_utf8 where arrays like

┌──────┐ ┌──────┐             ┌──────┐
│ foo  │ │ aaa  │             │fooaaa│
├──────┤ ├──────┤  ────────▶  ├──────┤
│<null>│ │ bbb  │             │<null>│
├──────┤ ├──────┤             ├──────┤
│ baz  │ │ ccc  │             │bazccc│
└──────┘ └──────┘             └──────┘

Describe the solution you'd like
I would like a function like:

fn concat_elements_dyn(left: &dyn Array, right: &dyn Array) -> ArrayRef {
}

That dispatches on type of arrays:

  1. If the types are different, it will return an error
  2. if the types are DataType::Utf8 it will invoke concat_elements_utf8

It would also be awesome to add support for other data types such as DataType::LargeUtf8, DataType::Binary, and DataType::LargeBinary

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
See #1747 and the original PR #1720 from @Ismail-Maj

@alamb alamb added arrow Changes to the arrow crate enhancement Any new improvement worthy of a entry in the changelog good first issue Good for newcomers labels May 26, 2022
@ismailmaj
Copy link
Contributor

Might as well continue :D Can I get assigned please?

@alamb
Copy link
Contributor Author

alamb commented May 31, 2022

Done!

@Weijun-H
Copy link
Contributor

Hi there, is this feature done? If not, I want to pick it up.

@alamb
Copy link
Contributor Author

alamb commented Feb 25, 2023

Hi @Weijun-H -- thanks!

I do not think it is done yet
https://docs.rs/arrow/33.0.0/arrow/compute/kernels/concat_elements/index.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate enhancement Any new improvement worthy of a entry in the changelog good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants