Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI][C++] Flaky DatasetWriterTestFixture.MaxRowsOneWriteBackpresure test #39326

Closed
mapleFU opened this issue Dec 20, 2023 · 3 comments · Fixed by #39379
Closed

[CI][C++] Flaky DatasetWriterTestFixture.MaxRowsOneWriteBackpresure test #39326

mapleFU opened this issue Dec 20, 2023 · 3 comments · Fixed by #39379
Assignees
@mapleFU
Labels
Component: C++ Component: Continuous Integration Type: enhancement
Milestone
15.0.0

Comments

@mapleFU
Copy link
Member

mapleFU commented Dec 20, 2023

Describe the enhancement requested

As mentioned in #38885 (comment) , the DatasetWriterTestFixture.MaxRowsOneWriteBackpresure might failed. This might because:

  1. In CI env, the file operation would be slow, see: GH-38884: [C++] DatasetWriter release rows_in_flight_throttle when allocate writing failed #38885 (comment)
  2. Maybe the execution has potential deadlock, see: GH-38884: [C++] DatasetWriter release rows_in_flight_throttle when allocate writing failed #38885 (comment)

I'll ensure if (2) has problem. If not, I'll just make test runs faster ( About (1), The assertion will wait for 64secs, which might be slow)

Component(s)

C++, Continuous Integration
@kou kou mentioned this issue Dec 21, 2023
Merged
@davisusanibar
Copy link
Contributor

Hello @mapleFU, I would appreciate it if you could clarify this issue for me. Do you consider this issue to be a blocker for next release v15.0.0 or is it possible to skip this test at the moment?

@mapleFU
Copy link
Member Author

mapleFU commented Dec 26, 2023

I'll fix it before v15.0.0, it's ok to become a blocker or not :-)

@github-actions github-actions bot mentioned this issue Dec 26, 2023
Merged
@mapleFU
Copy link
Member Author

mapleFU commented Dec 26, 2023

I think (2) can hardly happens, because I didn't see any blocking wait called. Just fixing (1) here.

mapleFU added a commit that referenced this issue Dec 29, 2023
@mapleFU
GH-39326: [C++] Flaky DatasetWriterTestFixture.MaxRowsOneWriteBackpre…
7c3480e 
…sure test (#39379)

### Rationale for this change

This patch reduce the number of open files in testing first. I've verify the test in 14.0.2, it hangs forever.

### What changes are included in this PR?

Change the test file number from 100 to 20

### Are these changes tested?

Already

### Are there any user-facing changes?

no

* Closes: #39326

Authored-by: mwish <maplewish117@gmail.com>
Signed-off-by: mwish <maplewish117@gmail.com>
@mapleFU mapleFU added this to the 15.0.0 milestone Dec 29, 2023
clayburn pushed a commit to clayburn/arrow that referenced this issue Jan 23, 2024
@mapleFU @clayburn
apacheGH-39326: [C++] Flaky DatasetWriterTestFixture.MaxRowsOneWriteB…
5cee222 
…ackpresure test (apache#39379)

### Rationale for this change

This patch reduce the number of open files in testing first. I've verify the test in 14.0.2, it hangs forever.

### What changes are included in this PR?

Change the test file number from 100 to 20

### Are these changes tested?

Already

### Are there any user-facing changes?

no

* Closes: apache#39326

Authored-by: mwish <maplewish117@gmail.com>
Signed-off-by: mwish <maplewish117@gmail.com>
dgreiss pushed a commit to dgreiss/arrow that referenced this issue Feb 19, 2024
@mapleFU @dgreiss
apacheGH-39326: [C++] Flaky DatasetWriterTestFixture.MaxRowsOneWriteB…
06a88ee 
…ackpresure test (apache#39379)

### Rationale for this change

This patch reduce the number of open files in testing first. I've verify the test in 14.0.2, it hangs forever.

### What changes are included in this PR?

Change the test file number from 100 to 20

### Are these changes tested?

Already

### Are there any user-facing changes?

no

* Closes: apache#39326

Authored-by: mwish <maplewish117@gmail.com>
Signed-off-by: mwish <maplewish117@gmail.com>
zanmato1984 pushed a commit to zanmato1984/arrow that referenced this issue Feb 28, 2024
@mapleFU @zanmato1984
apacheGH-39326: [C++] Flaky DatasetWriterTestFixture.MaxRowsOneWriteB…
1d97265 
…ackpresure test (apache#39379)

### Rationale for this change

This patch reduce the number of open files in testing first. I've verify the test in 14.0.2, it hangs forever.

### What changes are included in this PR?

Change the test file number from 100 to 20

### Are these changes tested?

Already

### Are there any user-facing changes?

no

* Closes: apache#39326

Authored-by: mwish <maplewish117@gmail.com>
Signed-off-by: mwish <maplewish117@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mapleFU mapleFU

Labels
Component: C++ Component: Continuous Integration Type: enhancement
Projects
None yet
Milestone
15.0.0
Development

Successfully merging a pull request may close this issue.

GH-39326: [C++] Flaky DatasetWriterTestFixture.MaxRowsOneWriteBackpresure test mapleFU/arrow
2 participants
@davisusanibar @mapleFU