-
Notifications
You must be signed in to change notification settings - Fork 374
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to use deltalake Schema
in write_deltalake
#1862
Comments
take |
I'm not actually looking into this, maybe I will later, just tried the |
I wouldn't call this a bug as the |
@r3stl355 I think there is a method call to convert delta schema to pyarrow, so perhaps we should allow take both inputs |
yep @ion-elgreco , |
@r3stl355 ah my bad, read too fast over your post |
take |
I missed the fact that the nullable and the metadata are stored in the fields attributes of the `pyarrow.Schema. Should I close this issue or is it still pertinent ? |
I don't know, maybe remove a |
@r3stl355 yes makes sense to add |
# Description A second attempt to extend the write_deltalake to accept either PyArrow or Deltalake schema (messed up the previous PR with some rebase issues) Added a test # Related Issue(s) closes #1862 --------- Signed-off-by: Nikolay Ulmasov <ulmasov@hotmail.com>
# Description A second attempt to extend the write_deltalake to accept either PyArrow or Deltalake schema (messed up the previous PR with some rebase issues) Added a test # Related Issue(s) closes delta-io#1862 --------- Signed-off-by: Nikolay Ulmasov <ulmasov@hotmail.com>
Environment
Delta-rs version: 0.13.0
Binding: Python 3.11
OS: Windows 10 WSL2 (Ubuntu 22.04.02 LTS)
Bug
I want to use a
deltalake.schema.Schema
instance in the functionwrite_deltalake(schema=...)
in order to control the nullable parameter and add metadata. However, apyarrow.lib.Schema
type is expected instead ofdeltalake._internal.Schema
. This is issue is reproducible with Polars and Pandas as shown with the snippet belowThe text was updated successfully, but these errors were encountered: