Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PHOENIX-6032: When phoenix.allow.system.catalog.rollback=true, a view still sees data from a column that was dropped #949

Merged
merged 1 commit into from
Nov 4, 2020

Conversation

ChinmaySKulkarni
Copy link
Contributor

No description provided.

@ChinmaySKulkarni
Copy link
Contributor Author

Please review @yanxinyi @jpisaac @gjacoby126 @twdsilva

@stoty
Copy link
Contributor

stoty commented Oct 30, 2020

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 5s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.x Compile Tests _
+1 💚 mvninstall 11m 41s 4.x passed
+1 💚 compile 1m 0s 4.x passed
+1 💚 checkstyle 1m 4s 4.x passed
+1 💚 javadoc 0m 46s 4.x passed
+0 🆗 spotbugs 3m 9s phoenix-core in 4.x has 953 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚 mvninstall 6m 15s the patch passed
+1 💚 compile 0m 59s the patch passed
+1 💚 javac 0m 59s the patch passed
-1 ❌ checkstyle 1m 4s phoenix-core: The patch generated 4 new + 2127 unchanged - 0 fixed = 2131 total (was 2127)
-1 ❌ whitespace 0m 0s The patch 7 line(s) with tabs.
+1 💚 javadoc 0m 45s the patch passed
+1 💚 spotbugs 3m 21s the patch passed
_ Other Tests _
-1 ❌ unit 171m 22s phoenix-core in the patch failed.
+1 💚 asflicense 0m 28s The patch does not generate ASF License warnings.
205m 29s
Reason Tests
Failed junit tests phoenix.end2end.UpsertSelectIT
phoenix.end2end.SkipScanQueryIT
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #949
JIRA Issue PHOENIX-6032
Optional Tests dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
uname Linux af08208ec8c5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev/phoenix-personality.sh
git revision 4.x / 2aaf2e2
Default Java Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstyle https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/1/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
whitespace https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/1/artifact/yetus-general-check/output/whitespace-tabs.txt
unit https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/1/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Results https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/1/testReport/
Max. process+thread count 6075 (vs. ulimit of 30000)
modules C: phoenix-core U: phoenix-core
Console output https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/1/console
versions git=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@stoty
Copy link
Contributor

stoty commented Oct 30, 2020

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 6s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.x Compile Tests _
+1 💚 mvninstall 12m 3s 4.x passed
+1 💚 compile 1m 0s 4.x passed
+1 💚 checkstyle 1m 5s 4.x passed
+1 💚 javadoc 0m 47s 4.x passed
+0 🆗 spotbugs 3m 8s phoenix-core in 4.x has 953 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚 mvninstall 6m 16s the patch passed
+1 💚 compile 0m 58s the patch passed
+1 💚 javac 0m 58s the patch passed
-1 ❌ checkstyle 1m 5s phoenix-core: The patch generated 4 new + 2127 unchanged - 0 fixed = 2131 total (was 2127)
-1 ❌ whitespace 0m 0s The patch 7 line(s) with tabs.
+1 💚 javadoc 0m 44s the patch passed
+1 💚 spotbugs 3m 21s the patch passed
_ Other Tests _
-1 ❌ unit 174m 14s phoenix-core in the patch failed.
+1 💚 asflicense 0m 29s The patch does not generate ASF License warnings.
208m 43s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #949
JIRA Issue PHOENIX-6032
Optional Tests dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
uname Linux b9bf4f9c9ca0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev/phoenix-personality.sh
git revision 4.x / 2aaf2e2
Default Java Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstyle https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/2/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
whitespace https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/2/artifact/yetus-general-check/output/whitespace-tabs.txt
unit https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/2/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Results https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/2/testReport/
Max. process+thread count 5842 (vs. ulimit of 30000)
modules C: phoenix-core U: phoenix-core
Console output https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/2/console
versions git=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@ChinmaySKulkarni
Copy link
Contributor Author

Fixed whitespace and checkStyle warnings. Also, test failures are not failing locally.

@stoty
Copy link
Contributor

stoty commented Oct 30, 2020

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 0s Docker mode activated.
-1 ❌ patch 0m 4s #949 does not apply to 4.x. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.
Subsystem Report/Notes
GITHUB PR #949
JIRA Issue PHOENIX-6032
Console output https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/3/console
versions git=2.17.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@ChinmaySKulkarni
Copy link
Contributor Author

Thanks for the +1 Xinyi. Any one else want to review this patch? @virajjasani @gjacoby126 ?

@ChinmaySKulkarni
Copy link
Contributor Author

Deleted last build to avoid noise. Waiting for latest build

@stoty
Copy link
Contributor

stoty commented Oct 31, 2020

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 29s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.x Compile Tests _
+1 💚 mvninstall 10m 39s 4.x passed
+1 💚 compile 0m 55s 4.x passed
+1 💚 checkstyle 1m 21s 4.x passed
+1 💚 javadoc 0m 44s 4.x passed
+0 🆗 spotbugs 2m 52s phoenix-core in 4.x has 953 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚 mvninstall 5m 20s the patch passed
+1 💚 compile 0m 56s the patch passed
+1 💚 javac 0m 56s the patch passed
+1 💚 checkstyle 1m 24s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 javadoc 0m 43s the patch passed
+1 💚 spotbugs 3m 7s the patch passed
_ Other Tests _
-1 ❌ unit 124m 44s phoenix-core in the patch failed.
+1 💚 asflicense 0m 35s The patch does not generate ASF License warnings.
156m 30s
Reason Tests
Failed junit tests phoenix.end2end.OrphanViewToolIT
phoenix.end2end.index.MutableIndexExtendedIT
phoenix.end2end.ConcurrentMutationsExtendedIT
phoenix.end2end.NullIT
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #949
JIRA Issue PHOENIX-6032
Optional Tests dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
uname Linux 23e271dbea11 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev/phoenix-personality.sh
git revision 4.x / 7cbebb4
Default Java Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
unit https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/5/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Results https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/5/testReport/
Max. process+thread count 6990 (vs. ulimit of 30000)
modules C: phoenix-core U: phoenix-core
Console output https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/5/console
versions git=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@virajjasani virajjasani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few minor nits, +1 (non-binding) for source changes with my understanding

Copy link
Contributor

@gjacoby126 gjacoby126 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a few nits

testUtil.getHBaseAdmin().enableTable(systemCatalog);
} catch (DoNotRetryIOException e) {
// table is not splittable
assert (e.getMessage().contains("NOT splittable"));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we require that we get this exception? If so should be an Assert.fail() after enableTable

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, should use JUnit assertions rather than Java assert keyword

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I know this is a copied test, but good to do the simple fixes while we're at it)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm I'm not sure. This diff is just a result of copy pasting from the original test SystemCatalogIT Let me try to find out. Will change to using JUnit assert.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like that test wasn't reliably trying to split SYSCAT at all. I will open a follow-up Jira to fix this since it is unrelated to 6032.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

… still sees data from a column that was dropped
@ChinmaySKulkarni
Copy link
Contributor Author

Thanks for your reviews @virajjasani @gjacoby126 I have addressed all your review comments. Can I get a +1?

@stoty
Copy link
Contributor

stoty commented Nov 3, 2020

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 5m 20s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.x Compile Tests _
+1 💚 mvninstall 11m 44s 4.x passed
+1 💚 compile 1m 0s 4.x passed
+1 💚 checkstyle 1m 5s 4.x passed
+1 💚 javadoc 0m 47s 4.x passed
+0 🆗 spotbugs 3m 9s phoenix-core in 4.x has 953 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚 mvninstall 6m 11s the patch passed
+1 💚 compile 1m 0s the patch passed
+1 💚 javac 1m 0s the patch passed
-1 ❌ checkstyle 1m 5s phoenix-core: The patch generated 1 new + 2141 unchanged - 1 fixed = 2142 total (was 2142)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 javadoc 0m 46s the patch passed
+1 💚 spotbugs 3m 22s the patch passed
_ Other Tests _
-1 ❌ unit 213m 12s phoenix-core in the patch failed.
+1 💚 asflicense 0m 45s The patch does not generate ASF License warnings.
253m 7s
Reason Tests
Failed junit tests phoenix.end2end.index.GlobalMutableTxIndexIT
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/6/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #949
JIRA Issue PHOENIX-6032
Optional Tests dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
uname Linux a82b46abca17 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev/phoenix-personality.sh
git revision 4.x / a8a9699
Default Java Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstyle https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/6/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
unit https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/6/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Results https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/6/testReport/
Max. process+thread count 5434 (vs. ulimit of 30000)
modules C: phoenix-core U: phoenix-core
Console output https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/6/console
versions git=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@stoty
Copy link
Contributor

stoty commented Nov 3, 2020

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 4m 31s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.x Compile Tests _
+1 💚 mvninstall 10m 55s 4.x passed
+1 💚 compile 0m 57s 4.x passed
+1 💚 checkstyle 1m 22s 4.x passed
+1 💚 javadoc 0m 43s 4.x passed
+0 🆗 spotbugs 2m 51s phoenix-core in 4.x has 953 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚 mvninstall 5m 18s the patch passed
+1 💚 compile 0m 58s the patch passed
+1 💚 javac 0m 58s the patch passed
+1 💚 checkstyle 1m 24s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 javadoc 0m 41s the patch passed
+1 💚 spotbugs 3m 3s the patch passed
_ Other Tests _
-1 ❌ unit 135m 10s phoenix-core in the patch failed.
+1 💚 asflicense 0m 34s The patch does not generate ASF License warnings.
171m 9s
Reason Tests
Failed junit tests phoenix.end2end.TenantSpecificViewIndexIT
phoenix.end2end.RowValueConstructorOffsetIT
phoenix.end2end.AlterTableWithViewsIT
phoenix.util.CoprocessorHConnectionTableFactoryIT
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/7/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #949
JIRA Issue PHOENIX-6032
Optional Tests dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
uname Linux aea7b97e7e90 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev/phoenix-personality.sh
git revision 4.x / 8aa243d
Default Java Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
unit https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/7/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Results https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/7/testReport/
Max. process+thread count 6967 (vs. ulimit of 30000)
modules C: phoenix-core U: phoenix-core
Console output https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/7/console
versions git=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@gjacoby126 gjacoby126 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, thanks @ChinmaySKulkarni

Copy link
Contributor

@virajjasani virajjasani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@ChinmaySKulkarni ChinmaySKulkarni merged commit 11575c0 into apache:4.x Nov 4, 2020
@ChinmaySKulkarni ChinmaySKulkarni deleted the PHOENIX-6032 branch November 4, 2020 02:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants