ARROW-15665: [C++] Add error handling option to StrptimeOptions #12464

rok · 2022-02-18T15:41:53Z

This is to resolve ARROW-15665.

github-actions · 2022-02-18T15:42:14Z

https://issues.apache.org/jira/browse/ARROW-15665

rok · 2022-02-23T12:30:37Z

@pitrou I'm not sure what's causing this (Check failed: out.array()->buffers[0] Null bitmap deleted by kernel but can_write_into_slices = true). Any insight?
Would kernel preallocation settings help this or am I missing something else?

pitrou · 2022-02-28T17:29:41Z

can_write_into_slices means the kernel exec machinery won't let you redefine buffer addresses: you can write into the buffers given to you by the machinery but you are not allowed to point them elsewhere.

rok · 2022-03-17T04:06:12Z

@pitrou I think this is ready for review. I'm not sure why can_write_into_slices=true is required here, could you please check that I'm not doing something ill advised.

rok · 2022-03-17T04:22:36Z

@lidavidm I moved Strptime to scalar_temporal_unary.cc. Do you think that makes sense or better keep it in scalar_string_ascii.cc?

lidavidm

I think it's fine to remove it from scalar_string_ascii.cc.

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc

lidavidm · 2022-03-17T12:09:45Z

cpp/src/arrow/compute/api_scalar.h

  TimeUnit::type unit;
+  /// Raise on parsing errors


What is the behavior if we don't raise parse errors?

Indeed, we may want to make this more descriptive, e.g. bool error_is_null or something. Opinions?

Changed to error_is_null. How about error_returns_null?

cpp/src/arrow/compute/kernels/scalar_string_test.cc

lidavidm · 2022-03-17T12:12:14Z

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc

+  size_t cur = 0;
+  std::string zone = "";
+  while (cur < format.size() - 1) {
+    if (format[cur] == '%') {


Do we want to respect %% for escaping?

Switched to regex replace to do escaping. Feels a bit over engineered.

Regex is a bit much, how about just checking if the next char is '%' and advancing the string if so? (I think we do such things elsewhere)

Switched to a sequence counting approach.

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc

python/pyarrow/_compute.pyx

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc

cpp/src/arrow/compute/kernels/scalar_string_test.cc

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc

jorisvandenbossche · 2022-03-21T18:43:09Z

python/pyarrow/_compute.pyx

    """

-    def __init__(self, format, unit):
-        self._set_options(format, unit)
+    def __init__(self, format, unit, error_is_null=True):


This should be False for the default?

Can you also add a test for this default? (a test that checks an error is raised for the default options without explicitly passing error_is_null)

Ah sorry, it was a remainder from the previous raise_errors parameter. Changed and added a test.

lidavidm · 2022-03-22T19:29:40Z

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc

+    int64_t* out_data = out->GetMutableValues<int64_t>(1);
+
+    if (self.error_is_null) {
+      arrow::internal::BitmapWriter out_writer(out->buffers[0]->mutable_data(),


Hmm, we might need to test this more. I thought we merged an optimization where INTERSECTION wouldn't allocate a null bitmap if all inputs were non-null.

Hm, what would you recommend we test? Bigger arrays?

Arrays with no input nulls

Got it. Added bitmap allocation and tests. Could you please check if we're still missing something?

rok · 2022-03-23T15:20:02Z

@lidavidm thanks for the review. Could you take another look?

lidavidm

Thanks, LGTM overall, just a couple comments

lidavidm · 2022-03-24T12:09:55Z

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc

+    int64_t* out_data = out->GetMutableValues<int64_t>(1);
+
+    if (self.error_is_null) {
+      if (!in.MayHaveNulls()) {


nit: it might be better to directly check out->buffers[0] instead of relying on the optimization being performed

Switched to if (out->buffers[0] == nullptr).

ursabot · 2022-03-24T15:32:01Z

Benchmark runs are scheduled for baseline = 623a15e and contender = ddb663b. ddb663b is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Failed ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.34% ⬆️0.08%] test-mac-arm
[Failed ⬇️0.36% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.51% ⬆️0.04%] ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

github-actions bot added Component: C++ Component: Python Component: R labels Feb 18, 2022

rok force-pushed the ARROW-15665 branch 2 times, most recently from da7c0b5 to 6352488 Compare February 23, 2022 02:58

rok force-pushed the ARROW-15665 branch 6 times, most recently from 4df261b to a71ed14 Compare March 17, 2022 03:06

rok marked this pull request as ready for review March 17, 2022 03:52

rok force-pushed the ARROW-15665 branch from a71ed14 to 45c86ef Compare March 17, 2022 04:11

lidavidm reviewed Mar 17, 2022

View reviewed changes

rok force-pushed the ARROW-15665 branch 2 times, most recently from 33ce31a to 70d6b28 Compare March 17, 2022 20:37

lidavidm reviewed Mar 18, 2022

View reviewed changes

cpp/src/arrow/compute/kernels/scalar_string_test.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc Outdated Show resolved Hide resolved

rok force-pushed the ARROW-15665 branch from 70d6b28 to faa74cc Compare March 18, 2022 15:10

rok commented Mar 18, 2022

View reviewed changes

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc Outdated Show resolved Hide resolved

jorisvandenbossche reviewed Mar 21, 2022

View reviewed changes

rok force-pushed the ARROW-15665 branch from 8d07f87 to 6546d81 Compare March 22, 2022 13:11

rok added 4 commits March 22, 2022 19:09

Adding raise_errors to StrptimeOptions

b53a2e8

Some changes and kernel.can_write_into_slices=false

cb73a00

Make raise_errors=false default in R

0b9be59

Review feedback

66a2883

Using FirstTimeBitmapWriter

1b3f7bb

rok force-pushed the ARROW-15665 branch 3 times, most recently from e721a39 to 04202c9 Compare March 22, 2022 19:14

lidavidm reviewed Mar 22, 2022

View reviewed changes

Review feedback

b958a09

rok force-pushed the ARROW-15665 branch from 04202c9 to b958a09 Compare March 23, 2022 14:57

lidavidm approved these changes Mar 23, 2022

View reviewed changes

rok force-pushed the ARROW-15665 branch from baaab73 to 85b83e9 Compare March 24, 2022 02:53

Review feedback

06ab469

rok force-pushed the ARROW-15665 branch from 85b83e9 to 06ab469 Compare March 24, 2022 11:32

lidavidm approved these changes Mar 24, 2022

View reviewed changes

Review feedback

b4a6e75

rok requested a review from lidavidm March 24, 2022 15:18

lidavidm approved these changes Mar 24, 2022

View reviewed changes

lidavidm closed this in ddb663b Mar 24, 2022

asfimport mentioned this pull request Mar 30, 2022

[C++] Add error handling option to StrptimeOptions #20115

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ARROW-15665: [C++] Add error handling option to StrptimeOptions #12464

ARROW-15665: [C++] Add error handling option to StrptimeOptions #12464

rok commented Feb 18, 2022

github-actions bot commented Feb 18, 2022

rok commented Feb 23, 2022

pitrou commented Feb 28, 2022

rok commented Mar 17, 2022 •

edited

rok commented Mar 17, 2022

lidavidm left a comment

lidavidm Mar 17, 2022

pitrou Mar 17, 2022

rok Mar 17, 2022

lidavidm Mar 17, 2022

rok Mar 23, 2022

lidavidm Mar 23, 2022

rok Mar 24, 2022

jorisvandenbossche Mar 21, 2022

rok Mar 22, 2022

lidavidm Mar 22, 2022

rok Mar 23, 2022

lidavidm Mar 23, 2022

rok Mar 24, 2022

rok commented Mar 23, 2022

lidavidm left a comment

lidavidm Mar 24, 2022

rok Mar 24, 2022

ursabot commented Mar 24, 2022 •

edited

ARROW-15665: [C++] Add error handling option to StrptimeOptions #12464

ARROW-15665: [C++] Add error handling option to StrptimeOptions #12464

Conversation

rok commented Feb 18, 2022

github-actions bot commented Feb 18, 2022

rok commented Feb 23, 2022

pitrou commented Feb 28, 2022

rok commented Mar 17, 2022 • edited

rok commented Mar 17, 2022

lidavidm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rok commented Mar 23, 2022

lidavidm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ursabot commented Mar 24, 2022 • edited

rok commented Mar 17, 2022 •

edited

ursabot commented Mar 24, 2022 •

edited