Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support bag.to_dataframe when query planning is enabled #10961

Closed
b-phi opened this issue Feb 27, 2024 · 1 comment · Fixed by #10963
Closed

Support bag.to_dataframe when query planning is enabled #10961

b-phi opened this issue Feb 27, 2024 · 1 comment · Fixed by #10963
Assignees
Labels
needs triage Needs a response from a contributor

Comments

@b-phi
Copy link

b-phi commented Feb 27, 2024

As far as I can tell, dask bags can only be converted to dask dataframes when query planning is disabled. It would be great to support both query planning, and the flexibility of turning bags into dataframes. Currently calling to_dataframe with query planning enabled throws the following error.

df = bag.to_dataframe(columns=['line'])
File ".../python3.10/site-packages/dask/bag/core.py", line 1624, in to_dataframe
return dd.DataFrame(dsk, dfs.name, meta, divisions)
TypeError: FrameBase.init() takes 2 positional arguments but 5 were given

@github-actions github-actions bot added the needs triage Needs a response from a contributor label Feb 27, 2024
@phofl
Copy link
Collaborator

phofl commented Feb 27, 2024

Thx for the report, put up a pr. It will be included in the next release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs triage Needs a response from a contributor
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants