Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

array function allows different data types #3123

Closed
alamb opened this issue Aug 13, 2022 · 3 comments
Closed

array function allows different data types #3123

alamb opened this issue Aug 13, 2022 · 3 comments
Labels
bug Something isn't working

Comments

@alamb
Copy link
Contributor

alamb commented Aug 13, 2022

Describe the bug
Both Spark and Postgres allow arrays of elements that have the same type -- e.g. Spark: https://spark.apache.org/docs/2.1.0/api/java/index.html?org/apache/spark/sql/types/ArrayType.html

The current array() function in datafusion creates a FixedSizeList arrow array. However, the FixedSizedList in arrow is more general as it allows the elements to be any type

Datafusion currently doesn't coerce the types to a common type

To Reproduce

DataFusion CLI v10.0.0select array(1,2,'foo');
+--------------------------------------+
| array(Int64(1),Int64(2),Utf8("foo")) |
+--------------------------------------+
| [1, 2, foo]                          |
+--------------------------------------+

Expected behavior
An error, like in postgres:

               ^
alamb=# select array[1,2,'foo'];
ERROR:  invalid input syntax for type integer: "foo"

Additional context
Found while working on #3122

The array function was originally introduced by @jorgecarleitao in https://github.com/apache/arrow/pull/8102/files

@alamb alamb added the bug Something isn't working label Aug 13, 2022
@comphead
Copy link
Contributor

I will try to solve it

@comphead
Copy link
Contributor

@alamb looks like it was already fixed in #2194

DataFusion CLI v11.0.0
❯ select array[1, 2, 'foo'];
NotImplemented("Arrays with different types are not supported: {Int64, Utf8}")

@alamb
Copy link
Contributor Author

alamb commented Aug 31, 2022

👍 thanks @comphead -- not sure what I was thinking about

@alamb alamb closed this as completed Aug 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants