-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GH-35676: [MATLAB] Add an InferNulls
name-value pair for controlling null value inference during construction of arrow.array.Array
#35827
Conversation
Co-authored-by: Kevin Gurney <kgurney@mathworks.com>
2. Remove NullDetectionFcn nv-pair.
|
2. Move license header to the top of the file
I think I've addressed all the current feedback. Let me know if we feel anything else needs to be changed. Thanks! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
Benchmark runs are scheduled for baseline = 8878fc8 and contender = f44f768. f44f768 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
Rationale for this change
This change lets users control toggle the automatic null-value detection behavior. By default, values MATLAB considers to be missing (e.g.
NaN
fordouble
,<missing>
forstring
, andNaT
fordatetime
) will be treated asnull
values. Users can toggle this behavior on and off using theInferNulls
name-value pair.Example
We've only added this nv-pair to
arrow.array.Float64Array
for now. We'll add this nv-pair to the other types in a followup changelist.What changes are included in this PR?
InferNulls
name-value pair toarrow.array.Float64Array
.arrow.args.validateTypeAndShape
to remove duplicate validation code among the numeric classes.arrow.args.parseValidElements
that thearrow.array.<Type>Array
classes will be able to share for generating the logical mask of valid elements.Are these changes tested?
Yes, we added a test pointed called
InferNulls
to the test classtFloat64Array.m
.Are there any user-facing changes?
Yes, users can now control how
NaN
values are treated when creating anarrow.array.Float64Array
.Future Directions
mathworks/libmexclass
. We have a prototype to do this using status-like and result-like objects already pushed to a branch. Once this branch is merged with themain
branch ofmathworks/libmexclass
, we'll port it over.Notes
Thank you @kevingurney for all the help with this PR!
InferNulls
name-value pair for controlling null value inference during construction ofarrow.array.Array
#35676