Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PHOENIX-5601 Add a new coprocessor for PHOENIX_TTL - PhoenixTTLRegionObserver #955

Closed
wants to merge 4 commits into from
Closed

PHOENIX-5601 Add a new coprocessor for PHOENIX_TTL - PhoenixTTLRegionObserver #955

wants to merge 4 commits into from

Conversation

jpisaac
Copy link
Contributor

@jpisaac jpisaac commented Nov 4, 2020

No description provided.

@jpisaac
Copy link
Contributor Author

jpisaac commented Nov 4, 2020

@yanxinyi @ChinmaySKulkarni @kadirozde

@stoty
Copy link
Contributor

stoty commented Nov 4, 2020

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 6s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ 4.x Compile Tests _
+1 💚 mvninstall 11m 0s 4.x passed
+1 💚 compile 0m 55s 4.x passed
+1 💚 checkstyle 2m 25s 4.x passed
+1 💚 javadoc 0m 45s 4.x passed
+0 🆗 spotbugs 2m 54s phoenix-core in 4.x has 951 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚 mvninstall 5m 10s the patch passed
+1 💚 compile 0m 55s the patch passed
+1 💚 javac 0m 55s the patch passed
-1 ❌ checkstyle 2m 33s phoenix-core: The patch generated 302 new + 4586 unchanged - 101 fixed = 4888 total (was 4687)
-1 ❌ whitespace 0m 0s The patch 15 line(s) with tabs.
+1 💚 javadoc 0m 42s the patch passed
+1 💚 spotbugs 3m 3s the patch passed
_ Other Tests _
-1 ❌ unit 190m 37s phoenix-core in the patch failed.
-1 ❌ asflicense 0m 37s The patch generated 3 ASF License warnings.
225m 31s
Reason Tests
Failed junit tests phoenix.end2end.IndexRepairRegionScannerIT
phoenix.end2end.DerivedTableIT
phoenix.end2end.index.DropColumnIT
phoenix.end2end.StatsEnabledSplitSystemCatalogIT
phoenix.end2end.index.IndexWithTableSchemaChangeIT
phoenix.end2end.AlterTableWithViewsIT
phoenix.end2end.DropIndexedColsIT
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-955/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #955
Optional Tests dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
uname Linux fe059a6bb2b7 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev/phoenix-personality.sh
git revision 4.x / b84121d
Default Java Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstyle https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-955/1/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
whitespace https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-955/1/artifact/yetus-general-check/output/whitespace-tabs.txt
unit https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-955/1/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Results https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-955/1/testReport/
asflicense https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-955/1/artifact/yetus-general-check/output/patch-asflicense-problems.txt
Max. process+thread count 6165 (vs. ulimit of 30000)
modules C: phoenix-core U: phoenix-core
Console output https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-955/1/console
versions git=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

ChinmaySKulkarni
ChinmaySKulkarni requested changes Nov 9, 2020
View reviewed changes
Copy link
Contributor

@ChinmaySKulkarni ChinmaySKulkarni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpisaac Added some comments and questions. Please take a look. Overall looks great!

phoenix-core/src/it/java/org/apache/phoenix/end2end/ViewTTLIT.java Outdated Show resolved Hide resolved
phoenix-core/src/it/java/org/apache/phoenix/end2end/ViewTTLIT.java Outdated Show resolved Hide resolved
phoenix-core/src/it/java/org/apache/phoenix/end2end/ViewTTLIT.java Outdated Show resolved Hide resolved
phoenix-core/src/it/java/org/apache/phoenix/end2end/ViewTTLIT.java Outdated Show resolved Hide resolved
phoenix-core/src/it/java/org/apache/phoenix/end2end/ViewTTLIT.java Outdated Show resolved Hide resolved
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixTTLRegionObserver.java Outdated Show resolved Hide resolved
phoenix-core/src/main/java/org/apache/phoenix/util/ScanUtil.java Outdated Show resolved Hide resolved
phoenix-core/src/main/java/org/apache/phoenix/util/ScanUtil.java Show resolved Hide resolved
phoenix-core/src/test/java/org/apache/phoenix/query/PhoenixTestBuilder.java Outdated Show resolved Hide resolved
phoenix-core/src/test/java/org/apache/phoenix/query/PhoenixTestBuilder.java Outdated Show resolved Hide resolved
@jpisaac
Copy link
Contributor Author

jpisaac commented Nov 11, 2020

@yanxinyi @ChinmaySKulkarni
Addressed most of the review comments
One task remaining is to add unit tests for ScanUtil utility functions

@stoty
Copy link
Contributor

stoty commented Nov 12, 2020

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 14s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ 4.x Compile Tests _
+1 💚 mvninstall 11m 3s 4.x passed
+1 💚 compile 1m 1s 4.x passed
+1 💚 checkstyle 2m 10s 4.x passed
+1 💚 javadoc 0m 47s 4.x passed
+0 🆗 spotbugs 3m 12s phoenix-core in 4.x has 946 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚 mvninstall 5m 34s the patch passed
+1 💚 compile 1m 0s the patch passed
+1 💚 javac 1m 0s the patch passed
-1 ❌ checkstyle 2m 10s phoenix-core: The patch generated 261 new + 4632 unchanged - 58 fixed = 4893 total (was 4690)
-1 ❌ whitespace 0m 0s The patch 15 line(s) with tabs.
+1 💚 javadoc 0m 46s the patch passed
+1 💚 spotbugs 3m 20s the patch passed
_ Other Tests _
-1 ❌ unit 195m 21s phoenix-core in the patch failed.
+1 💚 asflicense 0m 38s The patch does not generate ASF License warnings.
231m 17s
Reason Tests
Failed junit tests phoenix.end2end.index.DropColumnIT
phoenix.end2end.StatsEnabledSplitSystemCatalogIT
phoenix.end2end.AlterTableWithViewsIT
phoenix.end2end.SystemCatalogRollbackEnabledIT
phoenix.end2end.DropIndexedColsIT
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-955/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #955
Optional Tests dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
uname Linux 5d1c53b2b132 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev/phoenix-personality.sh
git revision 4.x / 565b0ea
Default Java Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstyle https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-955/3/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
whitespace https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-955/3/artifact/yetus-general-check/output/whitespace-tabs.txt
unit https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-955/3/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Results https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-955/3/testReport/
Max. process+thread count 6226 (vs. ulimit of 30000)
modules C: phoenix-core U: phoenix-core
Console output https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-955/3/console
versions git=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

ChinmaySKulkarni
ChinmaySKulkarni reviewed Nov 13, 2020
View reviewed changes
phoenix-core/src/it/java/org/apache/phoenix/end2end/ViewTTLIT.java Show resolved Hide resolved
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixTTLRegionObserver.java Show resolved Hide resolved
...main/java/org/apache/phoenix/coprocessor/metrics/MetricsPhoenixCoprocessorSourceFactory.java Show resolved Hide resolved
...main/java/org/apache/phoenix/coprocessor/metrics/MetricsPhoenixCoprocessorSourceFactory.java Outdated
return INSTANCE;
}

public synchronized MetricsPhoenixTTLSource getPhoenixTTLSource() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpisaac can we address this please?

phoenix-core/src/test/java/org/apache/phoenix/util/ScanUtilTest.java
@@ -466,4 +482,78 @@ public SortOrder getSortOrder() {
assertArrayEquals(expectedEndKey, endKey);
}
}

public static class PhoenixTTLScanUtilTest extends BaseConnectionlessQueryTest {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice tests @jpisaac !

ChinmaySKulkarni
ChinmaySKulkarni approved these changes Nov 16, 2020
View reviewed changes
Copy link
Contributor

@ChinmaySKulkarni ChinmaySKulkarni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

yanxinyi
yanxinyi reviewed Nov 16, 2020
View reviewed changes
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
if (!SchemaUtil.isSystemTable(tableName)) {
if (!descriptor.hasCoprocessor(PhoenixTTLRegionObserver.class.getName())) {
descriptor.addCoprocessor(
PhoenixTTLRegionObserver.class.getName(), null, priority-2, null);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why priority - 2 here?

@gjacoby126
Copy link
Contributor

JIRA is resolved, closing PR

@gjacoby126 gjacoby126 closed this Dec 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yanxinyi yanxinyi yanxinyi left review comments

@ChinmaySKulkarni ChinmaySKulkarni ChinmaySKulkarni approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
5 participants
@jpisaac @stoty @gjacoby126 @ChinmaySKulkarni @yanxinyi