Skip to content

Comments

[SPARK-44809][SS] Remove unused RocksDB custom metrics for pause/writeBatch#42491

Closed
anishshri-db wants to merge 1 commit intoapache:masterfrom
anishshri-db:task/SPARK-44809
Closed

[SPARK-44809][SS] Remove unused RocksDB custom metrics for pause/writeBatch#42491
anishshri-db wants to merge 1 commit intoapache:masterfrom
anishshri-db:task/SPARK-44809

Conversation

@anishshri-db
Copy link
Contributor

@anishshri-db anishshri-db commented Aug 14, 2023

What changes were proposed in this pull request?

Remove unused RocksDB custom metrics for pause/writeBatch

Why are the changes needed?

Remove unused metrics from query progress

Does this PR introduce any user-facing change?

No

How was this patch tested?

Modified existing tests

[info] Run completed in 23 seconds, 724 milliseconds.
[info] Total number of tests run: 11
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 11, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 53 s, completed Aug 14, 2023, 3:30:56 PM

@anishshri-db anishshri-db changed the title [SPARK-44809] Removed unused RocksDB custom metrics for pause/writeBatch [SPARK-44809] Remove unused RocksDB custom metrics for pause/writeBatch Aug 14, 2023
@anishshri-db anishshri-db changed the title [SPARK-44809] Remove unused RocksDB custom metrics for pause/writeBatch [SPARK-44809][SS] Remove unused RocksDB custom metrics for pause/writeBatch Aug 14, 2023
@anishshri-db
Copy link
Contributor Author

cc - @HeartSaVioR - PTAL, thx !

@anishshri-db
Copy link
Contributor Author

I don't think error is related to the change. Seems like a transient/timing issue with awaitOffset. Locally, tests run fine. Ran suite multiple times

[info] Run completed in 31 seconds, 40 milliseconds.
[info] Total number of tests run: 8
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 8, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.

Copy link
Contributor

@HeartSaVioR HeartSaVioR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@HeartSaVioR
Copy link
Contributor

@anishshri-db Can we retry the build for completeness' sake?

@anishshri-db
Copy link
Contributor Author

@HeartSaVioR
Copy link
Contributor

https://github.com/anishshri-db/spark/actions/runs/5861353460/attempts/2 <= This passed.

Thanks! Merging to master.

valentinp17 pushed a commit to valentinp17/spark that referenced this pull request Aug 24, 2023
…eBatch

### What changes were proposed in this pull request?
Remove unused RocksDB custom metrics for pause/writeBatch

### Why are the changes needed?
Remove unused metrics from query progress

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Modified existing tests

```
[info] Run completed in 23 seconds, 724 milliseconds.
[info] Total number of tests run: 11
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 11, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 53 s, completed Aug 14, 2023, 3:30:56 PM
```

Closes apache#42491 from anishshri-db/task/SPARK-44809.

Authored-by: Anish Shrigondekar <anish.shrigondekar@databricks.com>
Signed-off-by: Jungtaek Lim <kabhwan.opensource@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants