PHOENIX-6086 : Take snapshot of all SYSTEM tables before attempting to upgrade them #952

virajjasani · 2020-11-02T06:48:01Z

No description provided.

stoty · 2020-11-02T10:18:22Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 9s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ master Compile Tests _
+1 💚	mvninstall	14m 9s	master passed
+1 💚	compile	1m 2s	master passed
+1 💚	checkstyle	1m 26s	master passed
+1 💚	javadoc	0m 46s	master passed
+0 🆗	spotbugs	3m 6s	phoenix-core in master has 969 extant spotbugs warnings.
		_ Patch Compile Tests _
+1 💚	mvninstall	8m 56s	the patch passed
+1 💚	compile	0m 58s	the patch passed
+1 💚	javac	0m 58s	the patch passed
-1 ❌	checkstyle	1m 27s	phoenix-core: The patch generated 30 new + 3476 unchanged - 26 fixed = 3506 total (was 3502)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	0m 46s	the patch passed
-1 ❌	spotbugs	3m 23s	phoenix-core generated 1 new + 963 unchanged - 6 fixed = 964 total (was 969)
		_ Other Tests _
-1 ❌	unit	169m 46s	phoenix-core in the patch failed.
+1 💚	asflicense	0m 23s	The patch does not generate ASF License warnings.
		209m 50s

Reason	Tests
FindBugs	module:phoenix-core
	org.apache.phoenix.query.ConnectionQueryServicesImpl.upgradeSystemTask(PhoenixConnection, Map) passes a nonconstant String to an execute or addBatch method on an SQL statement At ConnectionQueryServicesImpl.java:String to an execute or addBatch method on an SQL statement At ConnectionQueryServicesImpl.java:[line 4146]
Failed junit tests	phoenix.schema.stats.NamespaceEnabledStatsCollectorIT

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#952
Optional Tests	dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
uname	Linux bb2d64d25e32 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev/phoenix-personality.sh
git revision	master / `e828ef7`
Default Java	Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstyle	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/1/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
spotbugs	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/1/artifact/yetus-general-check/output/new-spotbugs-phoenix-core.html
unit	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/1/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Results	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/1/testReport/
Max. process+thread count	6199 (vs. ulimit of 30000)
modules	C: phoenix-core U: phoenix-core
Console output	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/1/console
versions	git=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

virajjasani · 2020-11-02T14:56:03Z

spotbug warning passes a nonconstant String to an execute or addBatch method on an SQL statement almost seem quite difficult to avoid (also not related to main code changes, it was updated due to other spotbug reason) since we need to provide arguments to Alter Table query.

virajjasani · 2020-11-03T14:56:50Z

NamespaceEnabledStatsCollectorIT seems flaky, the failure is not relevant to the changes.

yanxinyi · 2020-11-03T23:51:34Z

phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java

+                        createSnapshot(snapshotName, SYSTEM_CATALOG_NAME);
+                        systemTableToSnapshotMap.put(SYSTEM_CATALOG_NAME,
+                            snapshotName);
+                        LOGGER.debug("Created snapshot for {}", SYSTEM_CATALOG_NAME);


nit: we should change the logging level from debug to info since this rarely happens, and we need this information for sure.

…o upgrade them

stoty · 2020-11-04T07:32:04Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	patch	0m 3s	#952 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
GITHUB PR	#952
Console output	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/2/console
versions	git=2.17.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…master

stoty · 2020-11-04T11:12:23Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	6m 2s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ master Compile Tests _
+1 💚	mvninstall	14m 39s	master passed
+1 💚	compile	1m 1s	master passed
+1 💚	checkstyle	1m 27s	master passed
+1 💚	javadoc	0m 49s	master passed
+0 🆗	spotbugs	3m 7s	phoenix-core in master has 967 extant spotbugs warnings.
-0 ⚠️	patch	3m 14s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+1 💚	mvninstall	8m 55s	the patch passed
+1 💚	compile	0m 56s	the patch passed
+1 💚	javac	0m 56s	the patch passed
-1 ❌	checkstyle	1m 27s	phoenix-core: The patch generated 65 new + 3429 unchanged - 63 fixed = 3494 total (was 3492)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	0m 50s	the patch passed
-1 ❌	spotbugs	3m 31s	phoenix-core generated 1 new + 961 unchanged - 6 fixed = 962 total (was 967)
		_ Other Tests _
-1 ❌	unit	171m 24s	phoenix-core in the patch failed.
+1 💚	asflicense	0m 22s	The patch does not generate ASF License warnings.
		217m 6s

Reason	Tests
FindBugs	module:phoenix-core
	org.apache.phoenix.query.ConnectionQueryServicesImpl.upgradeSystemTask(PhoenixConnection, Map) passes a nonconstant String to an execute or addBatch method on an SQL statement At ConnectionQueryServicesImpl.java:String to an execute or addBatch method on an SQL statement At ConnectionQueryServicesImpl.java:[line 4134]

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#952
Optional Tests	dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
uname	Linux 41412c74f0e5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev/phoenix-personality.sh
git revision	master / `337d795`
Default Java	Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstyle	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/3/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
spotbugs	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/3/artifact/yetus-general-check/output/new-spotbugs-phoenix-core.html
unit	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/3/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Results	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/3/testReport/
Max. process+thread count	5259 (vs. ulimit of 30000)
modules	C: phoenix-core U: phoenix-core
Console output	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/3/console
versions	git=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

virajjasani · 2020-11-05T12:30:19Z

Thanks for the review @yanxinyi

…versions (apache#952) Align Phoenix's curator dependency imports with those of Hadoop and HBase, which also do some exclusions. - Exclude org.apache.zookeeper:* - Exclude org.slf4j:* - Exclude com.google.guava:guava from curator-client Reason: Build Test Plan: Build

yanxinyi reviewed Nov 3, 2020

View reviewed changes

yanxinyi approved these changes Nov 4, 2020

View reviewed changes

PHOENIX-6086 : Take snapshot of all SYSTEM tables before attempting t…

ae9359f

…o upgrade them

Merge branch 'master' of github.com:apache/phoenix into PHOENIX-6086-…

e5b7bcf

…master

virajjasani closed this Nov 5, 2020

virajjasani deleted the PHOENIX-6086-master branch November 5, 2020 12:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PHOENIX-6086 : Take snapshot of all SYSTEM tables before attempting to upgrade them #952

PHOENIX-6086 : Take snapshot of all SYSTEM tables before attempting to upgrade them #952

virajjasani commented Nov 2, 2020

stoty commented Nov 2, 2020

virajjasani commented Nov 2, 2020

virajjasani commented Nov 3, 2020

yanxinyi Nov 3, 2020

stoty commented Nov 4, 2020

stoty commented Nov 4, 2020

virajjasani commented Nov 5, 2020

PHOENIX-6086 : Take snapshot of all SYSTEM tables before attempting to upgrade them #952

PHOENIX-6086 : Take snapshot of all SYSTEM tables before attempting to upgrade them #952

Conversation

virajjasani commented Nov 2, 2020

stoty commented Nov 2, 2020

virajjasani commented Nov 2, 2020

virajjasani commented Nov 3, 2020

yanxinyi Nov 3, 2020

Choose a reason for hiding this comment

stoty commented Nov 4, 2020

stoty commented Nov 4, 2020

virajjasani commented Nov 5, 2020