BIGTOP-3987: Upgrade phoenix version to 5.1.3 #1181

Akanksha-kedia · 2023-09-25T14:39:23Z

Apache Phoenix is an add-on for Apache HBase that provides a programmatic ANSI SQL interface. Apache Phoenix implements best-practice optimizations to enable software engineers to develop next-generation data-driven applications based on HBase.

Hbase in Bigtop 3.3 is 2.5 that needs Phoenix 5.1.3 to support.

++ PHOENIX_VERSION=5.1.3
++ HBASE_VERSION=2.5.3
++ JDK_VERSION=1.8

export MAVEN_OPTS=-Xmx1024m
MAVEN_OPTS=-Xmx1024m
MVN_ARGS='-Phadoop-3.0 '
MVN_ARGS+='-Dhadoop-three.version=3.3.5 '
MVN_ARGS+='-Dhadoop.guava.version=27.0-jre '
MVN_ARGS+='-Djetty.version=9.3.29.v20201019 '
MVN_ARGS+='-Dzookeeper.version=3.5.9 '
MVN_ARGS+='-DskipTests '
MVN_ARGS+='-Dcheckstyle.skip=true '

Akanksha-kedia · 2023-09-25T15:01:52Z

@smallyao please review

sekikn · 2023-10-15T23:38:54Z

The current PR doesn't work as follows. Removing patch0-PHOENIX-6766-branch-5.1.diff (and some additional work on install_phoenix.sh) seem to be required.

$ curl -sL https://github.com/apache/bigtop/pull/1181.diff | git apply
$ git diff
diff --git a/bigtop.bom b/bigtop.bom
index 23e2a07..160666b 100644
--- a/bigtop.bom
+++ b/bigtop.bom
@@ -247,7 +247,7 @@ bigtop {
     'phoenix' {
       name    = 'phoenix'
       relNotes = 'Apache Phoenix: A SQL skin over HBase'
-      version { base = "5.1.2"; pkg = base; release = 1 }
+      version { base = "5.1.3"; pkg = base; release = 1 }
       tarball { destination = "$name-${version.base}-src.tar.gz"
                 source      = destination }
       url     { download_path = "/$name/$name-${version.base}"
$ ./gradlew phoenix-clean phoenix-pkg

...

Patch #0 (patch0-PHOENIX-6766-branch-5.1.diff):
+ /usr/bin/cat /home/vagrant/bigtop/build/phoenix/rpm/SOURCES/patch0-PHOENIX-6766-branch-5.1.diff
+ /usr/bin/patch -p1 --fuzz=0
patching file pom.xml
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file pom.xml.rej
error: Bad exit status from /var/tmp/rpm-tmp.NiKOV9 (%prep)
    Bad exit status from /var/tmp/rpm-tmp.NiKOV9 (%prep)


RPM build errors:

FAILURE: Build failed with an exception.

sekikn · 2023-10-18T00:13:41Z

@Akanksha-kedia Is there any progress on it? If you don't have enough time for now, may I take it over?

Akanksha-kedia · 2023-10-18T04:10:49Z

@sekikn i built with phenox 5.1.3 with hbase 2.5.3 and i was able without any error. Will try with this indiviually, just guide i ll try, Thanks for the concern.

Akanksha-kedia · 2023-10-18T04:11:06Z

Apache Phoenix is an add-on for Apache HBase that provides a programmatic ANSI SQL interface. Apache Phoenix implements best-practice optimizations to enable software engineers to develop next-generation data-driven applications based on HBase.

Hbase in Bigtop 3.3 is 2.5 that needs Phoenix 5.1.3 to support.

++ PHOENIX_VERSION=5.1.3 ++ HBASE_VERSION=2.5.3 ++ JDK_VERSION=1.8

export MAVEN_OPTS=-Xmx1024m MAVEN_OPTS=-Xmx1024m MVN_ARGS='-Phadoop-3.0 ' MVN_ARGS+='-Dhadoop-three.version=3.3.5 ' MVN_ARGS+='-Dhadoop.guava.version=27.0-jre ' MVN_ARGS+='-Djetty.version=9.3.29.v20201019 ' MVN_ARGS+='-Dzookeeper.version=3.5.9 ' MVN_ARGS+='-DskipTests ' MVN_ARGS+='-Dcheckstyle.skip=true '

@sekikn

Akanksha-kedia · 2023-10-18T04:43:27Z

[INFO] Apache HBase - Shaded - Testing Util Tester ........ SUCCESS [ 5.031 s]
[INFO] Apache HBase Shaded Packaging Invariants ........... SUCCESS [ 6.212 s]
[INFO] Apache HBase Shaded Packaging Invariants (with Hadoop bundled) FAILURE [ 4.116 s]
[INFO] Apache HBase - Archetypes .......................... SKIPPED
[INFO] Apache HBase - Exemplar for hbase-client archetype . SKIPPED
[INFO] Apache HBase - Exemplar for hbase-shaded-client archetype SKIPPED
[INFO] Apache HBase - Archetype builder ................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 05:56 min
[INFO] Finished at: 2023-10-18T04:33:32Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:exec (check-jar-contents-for-stuff-with-hadoop) on project hbase-shaded-with-hadoop-check-invariants: Command execution failed.: Process exited with an error: 1 (Exit value: 1) -> [Help 1]

sekikn · 2023-10-18T11:54:24Z

I'm afraid I can't give any meaningful advice from your last two comments, because it only describes your maven options and a part of the build results. Please provide your environment, branch/revision you're working on, code modification and executed commands. See the following examples.

Environment information:

$ facter os
{
  architecture => "amd64",
  distro => {
    codename => "jammy",
    description => "Ubuntu 22.04.3 LTS",
    id => "Ubuntu",
    release => {
      full => "22.04",
      major => "22.04"
    }
  },
  family => "Debian",
  hardware => "x86_64",
  name => "Ubuntu",
  release => {
    full => "22.04",
    major => "22.04"
  },
  selinux => {
    enabled => false
  }
}

Base branch/revision and code modification:

$ git rev-parse HEAD
f4d65ae97b484c8e01efa936948ed01574f149d2
$ git status 
On branch master
Your branch is ahead of 'origin/master' by 59 commits.
  (use "git push" to publish your local commits)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   bigtop.bom

no changes added to commit (use "git add" and/or "git commit -a")
$ git diff
diff --git a/bigtop.bom b/bigtop.bom
index 70ff2c1d..34ded091 100644
--- a/bigtop.bom
+++ b/bigtop.bom
@@ -247,7 +247,7 @@ bigtop {
     'phoenix' {
       name    = 'phoenix'
       relNotes = 'Apache Phoenix: A SQL skin over HBase'
-      version { base = "5.1.2"; pkg = base; release = 1 }
+      version { base = "5.1.3"; pkg = base; release = 1 }
       tarball { destination = "$name-${version.base}-src.tar.gz"
                 source      = destination }
       url     { download_path = "/$name/$name-${version.base}"

Executed command and its result containing error messages:

$ ./gradlew phoenix-clean phoenix-pkg

...

dpkg-source: error: LC_ALL=C patch -t -F 0 -N -p1 -u -V never -E -b -B .pc/patch0-PHOENIX-6766-branch-5.1.diff/ --reject-file=- < phoenix-5.1.3.orig.abmmQO/debian/patches/patch0-PHOENIX-6766-branch-5.1.diff subprocess returned exit status 1
patching file pom.xml
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored
dpkg-source: info: the patch has fuzz which is not allowed, or is malformed
dpkg-source: info: if patch 'patch0-PHOENIX-6766-branch-5.1.diff' is correctly applied by quilt, use 'quilt refresh' to update it
dpkg-source: info: if the file is present in the unpacked source, make sure it is also present in the orig tarball
dpkg-buildpackage: error: dpkg-source -b . subprocess returned exit status 2

FAILURE: Build failed with an exception.

* Where:
Script '/home/sekikn/repos/bigtop/packages.gradle' line: 456

* What went wrong:
Execution failed for task ':phoenix-sdeb'.
> Process 'command 'dpkg-buildpackage'' finished with non-zero exit value 2

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 7s
5 actionable tasks: 5 executed

Akanksha-kedia · 2023-10-24T08:41:50Z

I'm afraid I can't give any meaningful advice from your last two comments, because it only describes your maven options and a part of the build results. Please provide your environment, branch/revision you're working on, code modification and executed commands. See the following examples.

Environment information:

$ facter os
{
  architecture => "amd64",
  distro => {
    codename => "jammy",
    description => "Ubuntu 22.04.3 LTS",
    id => "Ubuntu",
    release => {
      full => "22.04",
      major => "22.04"
    }
  },
  family => "Debian",
  hardware => "x86_64",
  name => "Ubuntu",
  release => {
    full => "22.04",
    major => "22.04"
  },
  selinux => {
    enabled => false
  }
}

Base branch/revision and code modification:

$ git rev-parse HEAD
f4d65ae97b484c8e01efa936948ed01574f149d2
$ git status 
On branch master
Your branch is ahead of 'origin/master' by 59 commits.
  (use "git push" to publish your local commits)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   bigtop.bom

no changes added to commit (use "git add" and/or "git commit -a")
$ git diff
diff --git a/bigtop.bom b/bigtop.bom
index 70ff2c1d..34ded091 100644
--- a/bigtop.bom
+++ b/bigtop.bom
@@ -247,7 +247,7 @@ bigtop {
     'phoenix' {
       name    = 'phoenix'
       relNotes = 'Apache Phoenix: A SQL skin over HBase'
-      version { base = "5.1.2"; pkg = base; release = 1 }
+      version { base = "5.1.3"; pkg = base; release = 1 }
       tarball { destination = "$name-${version.base}-src.tar.gz"
                 source      = destination }
       url     { download_path = "/$name/$name-${version.base}"

Executed command and its result containing error messages:

$ ./gradlew phoenix-clean phoenix-pkg

...

dpkg-source: error: LC_ALL=C patch -t -F 0 -N -p1 -u -V never -E -b -B .pc/patch0-PHOENIX-6766-branch-5.1.diff/ --reject-file=- < phoenix-5.1.3.orig.abmmQO/debian/patches/patch0-PHOENIX-6766-branch-5.1.diff subprocess returned exit status 1
patching file pom.xml
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored
dpkg-source: info: the patch has fuzz which is not allowed, or is malformed
dpkg-source: info: if patch 'patch0-PHOENIX-6766-branch-5.1.diff' is correctly applied by quilt, use 'quilt refresh' to update it
dpkg-source: info: if the file is present in the unpacked source, make sure it is also present in the orig tarball
dpkg-buildpackage: error: dpkg-source -b . subprocess returned exit status 2

FAILURE: Build failed with an exception.

* Where:
Script '/home/sekikn/repos/bigtop/packages.gradle' line: 456

* What went wrong:
Execution failed for task ':phoenix-sdeb'.
> Process 'command 'dpkg-buildpackage'' finished with non-zero exit value 2

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 7s
5 actionable tasks: 5 executed

@Akanksha-kedia Is there any progress on it? If you don't have enough time for now, may I take it over?

please take over

sekikn · 2023-10-25T04:27:45Z

Thanks, I'll close this PR and submit another one once it's prepared.

BIGTOP-3987: Upgrade phoenix version to 5.1.3

9cf5dfc

sekikn closed this Oct 25, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BIGTOP-3987: Upgrade phoenix version to 5.1.3 #1181

BIGTOP-3987: Upgrade phoenix version to 5.1.3 #1181

Akanksha-kedia commented Sep 25, 2023

Akanksha-kedia commented Sep 25, 2023

sekikn commented Oct 15, 2023

sekikn commented Oct 18, 2023

Akanksha-kedia commented Oct 18, 2023

Akanksha-kedia commented Oct 18, 2023

Akanksha-kedia commented Oct 18, 2023

sekikn commented Oct 18, 2023

Akanksha-kedia commented Oct 24, 2023

sekikn commented Oct 25, 2023

BIGTOP-3987: Upgrade phoenix version to 5.1.3 #1181

BIGTOP-3987: Upgrade phoenix version to 5.1.3 #1181

Conversation

Akanksha-kedia commented Sep 25, 2023

Akanksha-kedia commented Sep 25, 2023

sekikn commented Oct 15, 2023

sekikn commented Oct 18, 2023

Akanksha-kedia commented Oct 18, 2023

Akanksha-kedia commented Oct 18, 2023

Akanksha-kedia commented Oct 18, 2023

sekikn commented Oct 18, 2023

Akanksha-kedia commented Oct 24, 2023

sekikn commented Oct 25, 2023