-
-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support empty structs #9216
Comments
Hi, does the team have a plan to support this? In a lot of cases, when parsing empty json columns from DB, the function panics. |
@sibarras Could you give a reproducible example of that panic? |
Sure, using sqlite, when you read a json column, it gets parsed as a str on polars. Then when you try to cast this to a struct, we got a panic. from sqlite3 import connect
import polars as pl
def main():
with connect(":memory:") as con:
df = pl.read_database(
"SELECT JSON('{}') as json_col;", con
) # it works fine, but it's parsed as a string
print(df)
df.select(pl.col("json_col").str.json_decode()) # panics here
if __name__ == "__main__":
main() This is the output using Python 3.9.18 on WSL2.
|
I looked into this and empty structs just don't really make much sense. An empty struct column would have to behave somewhat like a We should probably first address #3462 before implementing this.
|
The empty struct also creates issues in Polars already handles empty structs, but in an inconsistent way. And the inconsistency causes panic exceptions in more complex situations. import io
import polars as pl
frame = pl.read_ndjson(io.StringIO('{"id": 1, "empty_struct": {}, "list_of_empty_struct": [{}]}'))
print(frame)
for col_name, col_type in frame.schema.items():
print(f'{col_name:>20} {col_type}') Output:
The expected type of the "empty_struct" column would be |
Problem description
Although perhaps not extremely useful, we should allow structs without any fields for the sake of consistency.
In the current behaviour, Polars conjures up a single unnamed field of type
Null
:Trying to create an empty struct through the
struct
expression results in a PanicException:Desired behaviour would be:
The text was updated successfully, but these errors were encountered: