-
Notifications
You must be signed in to change notification settings - Fork 53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Polars: lazyframe_domain
ffi
#769
Conversation
This stack of pull requests is managed by Graphite. Learn more about stacking. Join @Shoeboxam and the rest of your teammates on Graphite |
edc1cd5
to
c79510c
Compare
c476811
to
bee219b
Compare
c79510c
to
edc1cd5
Compare
bee219b
to
c476811
Compare
edc1cd5
to
9e0b91e
Compare
c476811
to
768bd59
Compare
768bd59
to
a44d843
Compare
12060bf
to
b9d3e63
Compare
a44d843
to
6b149ef
Compare
b9d3e63
to
1d120c8
Compare
6b149ef
to
5ded19c
Compare
1d120c8
to
58011f8
Compare
5ded19c
to
de67fe7
Compare
ee4bb63
to
5a9100b
Compare
6a90c3d
to
ea4bf78
Compare
5a9100b
to
eac4eec
Compare
dp.series_domain("A", dp.atom_domain(T=dp.f64)), | ||
dp.series_domain("B", dp.atom_domain(T=dp.i32)), | ||
dp.series_domain("C", dp.option_domain(dp.atom_domain(T=dp.String))), | ||
dp.series_domain("D", dp.atom_domain(T=dp.i32)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the python types also work, it'd be good to test them. If they don't, I'd worry about usability -- Not necessarily something to address now, but a comment pointing to an issue would be good if it's not something for right now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
An annoying aspect is that Polars defaults to i64
, but the default inferred type of int
is i32
. So if you don't specify the types exactly, then the domains will each expect i32
but the data (a few lines below) is actually i64
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the explanation. It would be easy for users to get this wrong, so if we can't get around this, maybe just a test to make sure the error message is readable. I'll file a little issue for this.
"keys" => MarginPub::Keys, | ||
"lengths" => MarginPub::Lengths, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This corresponds to this test helper?
def _with_margin(domain, by):
return dp.with_margin(domain, by=by, public_info="lengths", max_partition_size=50)
I feel weird about passing in strings when we really just need to convey a boolean.
Also: It looks like "lengths" is exercised in the tests. Should we have some coverage for "keys"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's actually three states: no public info, known keys, and known lengths. I think the stringly-typed argument is clearer. An enum would be best, but I don't have FFI for it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added another test for initializing with keys.
ea4bf78
to
f63681f
Compare
d4322b3
to
ea54342
Compare
be70738
to
86809f6
Compare
8f37e85
to
918e69a
Compare
86809f6
to
5a5b2fd
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. Changes look good.
dp.series_domain("A", dp.atom_domain(T=dp.f64)), | ||
dp.series_domain("B", dp.atom_domain(T=dp.i32)), | ||
dp.series_domain("C", dp.option_domain(dp.atom_domain(T=dp.String))), | ||
dp.series_domain("D", dp.atom_domain(T=dp.i32)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the explanation. It would be easy for users to get this wrong, so if we can't get around this, maybe just a test to make sure the error message is readable. I'll file a little issue for this.
be5783c
to
222bb6b
Compare
5a5b2fd
to
568dd52
Compare
Closes #768