Skip to content

BUG: alter column type crash #6926

Description

@dentiny

Hi team, the following python script crashes

import tempfile
import shutil

import pyarrow as pa
import lance

def main() -> None:
    tmpdir = tempfile.mkdtemp(prefix="lance_nested_cast_")
    try:
        # Schema: { b: struct<c: int32> }, b non-nullable, c non-nullable.
        schema = pa.schema(
            [
                pa.field(
                    "b",
                    pa.struct([pa.field("c", pa.int32(), nullable=False)]),
                    nullable=False,
                )
            ]
        )

        table = pa.table(
            {
                "b": pa.StructArray.from_arrays(
                    [pa.array([1, 2, 3], type=pa.int32())],
                    fields=[pa.field("c", pa.int32(), nullable=False)],
                )
            },
            schema=schema,
        )

        ds = lance.write_dataset(table, tmpdir)
        print("initial schema:", ds.schema)

        ds.alter_columns(
            {"path": "b.c", "data_type": pa.int64()},
        )
        print("after alter schema:", ds.schema)
        print("data:", ds.to_table().to_pylist())
    finally:
        shutil.rmtree(tmpdir, ignore_errors=True)


if __name__ == "__main__":
    main()

error message

hjiang$ python3 bug.py
initial schema: b: struct<c: int32 not null> not null
child 0, c: int32 not null

thread 'lance_background_thread' (25255780) panicked at /Users/runner/work/lance/lance/rust/lance/src/dataset/schema_evolution.rs:637:39:
called Option::unwrap() on a None value
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
Traceback (most recent call last):
File "/Users/hjiang/Desktop/bug.py", line 51, in
main()
~~~~^^
File "/Users/hjiang/Desktop/bug.py", line 41, in main
ds.alter_columns(
~~~~~~~~~~~~~~~~^
{"path": "b.c", "data_type": pa.int64()},
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/opt/homebrew/lib/python3.14/site-packages/lance/dataset.py", line 1818, in alter_columns
self._ds.alter_columns(list(alterations))
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
RuntimeError: Task was aborted

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions