New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Schema: <schema> not found in DaskSQLContext #975
Comments
Thanks for raising this issue. We do have some limited schema support with the move over to datafusion and are working on expanding this to support the same level of operations as earlier. (See #841). @hungcs It would be great if you're able to provide a minimal example/reproducer of where schema support is failing in your workflow. |
@ayushdg sorry for the delay, heres my repo (still not working in 2022.3):
|
playing around with some stuff, i can register the schema by doing this:
am i doing something wrong? if i try
and it looks like it's trying to use |
Sorry for the delayed response here - could you share what dask-sql version you're using to reproduce these failures? With a source install of Lines 803 to 806 in 883cc3c
In general, would not recommend directly calling import dask_sql
import pandas as pd
context = dask_sql.Context()
connection_name = "file_uploads"
dataset_name = "titanic_dataset"
df = pd.DataFrame({"name": ["Tom"], "mask": ["pink"], "weapon": ["stick"]})
context.create_schema(connection_name)
context.create_table(dataset_name, df, schema_name=connection_name)
context.sql(f"use schema {connection_name}")
context.sql("select * from titanic_dataset").compute() I do understand why one wouldn't get the impression that the |
this worked in dask 2022.8, but after the switch to dataFusion, I get this error when running queries. We believe this is because dataFusion doesn't support schemas - is it possible to add support this again?
The text was updated successfully, but these errors were encountered: