-
-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove DataFrame assumptions from Expr #142
Comments
I think this makes sense doing now if we already run into other expressions. I can take a look at this. |
Just to make my current thinking concrete, I'm guessing that we have something like the following: class Expr:
def simplify(...)
def operands(...)
def __dask_graph__(...)
... def other things that have to do with graph manipulation
class Frame(Expr):
def divisions(...)
... other things that have to do with dataframe metadata
def groupby(...)
... other dataframe API methods Probably it gets more complicated once someone gets more deeply into this topic though. |
Yeah that was my understanding as well. I'll take a look and see whether I can come up with something useful. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Eventually we'll have all sorts of Exprs like arrays, bags, etc.. Today we're mostly focused on dataframe and that's good I think.
However, even today we have non-DataFrame exprs like scalars, and the results of
to_parquet
calls. Maybe it's time to pull out some of the dataframe metadata assumptions likedivisions
and maybe evenmeta
from theExpr
class and make some newFrame
class (or some better name) from which dataframe-like-things (dataframe, series, index) inherit.The text was updated successfully, but these errors were encountered: