Skip to content

BIGTOP-3504. Split the Phoenix package into client and server. #742

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
sekikn wants to merge 1 commit into
base: master
Choose a base branch
from
Open

BIGTOP-3504. Split the Phoenix package into client and server. #742

sekikn wants to merge 1 commit into from

Conversation

@sekikn
Copy link
Contributor

@sekikn sekikn commented Feb 23, 2021

@sekikn
Copy link
Contributor Author

sekikn commented Feb 23, 2021

I manually tested this PR on CentOS 7:

$ ./gradlew phoenix-clean phoenix-pkg repo

...

BUILD SUCCESSFUL in 11m 18s
9 actionable tasks: 9 executed
$ rpm -qlp output/phoenix/noarch/phoenix-5.1.0-1.el7.noarch.rpm 
/etc/default/phoenix
/usr/lib/phoenix/phoenix-pherf-5.1.0.jar
/usr/lib/phoenix/phoenix-server-hbase-2.2-5.1.0.jar
/usr/lib/phoenix/phoenix-server.jar
$ rpm -qlp output/phoenix/noarch/phoenix-client-5.1.0-1.el7.noarch.rpm 
/etc/phoenix/conf.dist
/usr/lib/phoenix/bin
/usr/lib/phoenix/bin/argparse-1.4.0
/usr/lib/phoenix/bin/argparse-1.4.0/argparse.py
/usr/lib/phoenix/bin/daemon.py
/usr/lib/phoenix/bin/end2endTest.py
/usr/lib/phoenix/bin/hadoop-metrics2-hbase.properties
/usr/lib/phoenix/bin/hadoop-metrics2-phoenix.properties
/usr/lib/phoenix/bin/hbase-omid-client-config.yml
/usr/lib/phoenix/bin/hbase-site.xml
/usr/lib/phoenix/bin/log4j.properties
/usr/lib/phoenix/bin/performance.py
/usr/lib/phoenix/bin/pherf-standalone.py
/usr/lib/phoenix/bin/phoenix_utils.py
/usr/lib/phoenix/bin/psql.py
/usr/lib/phoenix/bin/readme.txt
/usr/lib/phoenix/bin/sqlline.py
/usr/lib/phoenix/bin/traceserver.py
/usr/lib/phoenix/phoenix-client-hbase-2.2-5.1.0.jar
/usr/lib/phoenix/phoenix-client.jar
/usr/share/doc/phoenix-5.1.0
/usr/share/doc/phoenix-5.1.0/LICENSE
/usr/share/doc/phoenix-5.1.0/NOTICE
/usr/share/doc/phoenix-5.1.0/examples
/usr/share/doc/phoenix-5.1.0/examples/STOCK_SYMBOL.csv
/usr/share/doc/phoenix-5.1.0/examples/STOCK_SYMBOL.sql
/usr/share/doc/phoenix-5.1.0/examples/WEB_STAT.csv
/usr/share/doc/phoenix-5.1.0/examples/WEB_STAT.sql
/usr/share/doc/phoenix-5.1.0/examples/WEB_STAT_QUERIES.sql
$ cd provisioner/docker
$ ./docker-hadoop.sh -d -k hdfs,hbase -L -C config_centos-7.yaml -c 3

...

$ for i in $(seq 1 3); do
>   if [ $i -eq 1 ]; then ./docker-hadoop.sh --exec $i yum install -y phoenix-client; fi
>   ./docker-hadoop.sh --exec $i yum install -y phoenix
>   ./docker-hadoop.sh --exec $i ln -s /usr/lib/phoenix/phoenix-server.jar /usr/lib/hbase/lib/
>   ./docker-hadoop.sh --exec $i systemctl restart hbase-regionserver
>   if [ $i -eq 1 ]; then ./docker-hadoop.sh --exec $i systemctl restart hbase-master; fi
> done

...

$ ./docker-hadoop.sh --exec 1 /usr/lib/phoenix/bin/sqlline.py

...

Connected to: Phoenix (version 5.1)
Driver: PhoenixEmbeddedDriver (version 5.1)
Autocommit status: true
Transaction isolation: TRANSACTION_READ_COMMITTED
sqlline version 1.9.0
0: jdbc:phoenix:> CREATE TABLE t(c VARCHAR PRIMARY KEY);
No rows affected (1.566 seconds)
0: jdbc:phoenix:> UPSERT INTO t VALUES ('foo');
1 row affected (0.164 seconds)
0: jdbc:phoenix:> SELECT * FROM t;
+-----+
|  C  |
+-----+
| foo |
+-----+
1 row selected (0.077 seconds)

and Debian 10:

$ ./gradlew phoenix-clean phoenix-pkg repo

...

BUILD SUCCESSFUL in 29m 55s
9 actionable tasks: 9 executed
$ dpkg -c output/phoenix/phoenix_5.1.0-1_all.deb 
drwxr-xr-x root/root         0 2021-02-23 15:03 ./
drwxr-xr-x root/root         0 2021-02-23 15:03 ./etc/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./etc/default/
-rw-r--r-- root/root       993 2021-02-23 15:03 ./etc/default/phoenix
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/lib/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/lib/phoenix/
-rw-r--r-- root/root   3679575 2021-02-23 15:03 ./usr/lib/phoenix/phoenix-pherf-5.1.0.jar
-rw-r--r-- root/root  67085723 2021-02-23 15:03 ./usr/lib/phoenix/phoenix-server-hbase-2.2-5.1.0.jar
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/share/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/share/doc/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/share/doc/phoenix/
-rw-r--r-- root/root       131 2021-02-23 15:03 ./usr/share/doc/phoenix/changelog.Debian.gz
-rw-r--r-- root/root     24257 2021-02-05 08:27 ./usr/share/doc/phoenix/changelog.gz
-rw-r--r-- root/root       318 2021-02-23 15:03 ./usr/share/doc/phoenix/copyright
lrwxrwxrwx root/root         0 2021-02-23 15:03 ./usr/lib/phoenix/phoenix-server.jar -> phoenix-server-hbase-2.2-5.1.0.jar
$ dpkg -c output/phoenix/phoenix-client_5.1.0-1_all.deb 
drwxr-xr-x root/root         0 2021-02-23 15:03 ./
drwxr-xr-x root/root         0 2021-02-23 15:03 ./etc/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./etc/phoenix/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./etc/phoenix/conf.dist/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/lib/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/lib/phoenix/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/lib/phoenix/bin/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/lib/phoenix/bin/argparse-1.4.0/
-rw-r--r-- root/root     89214 2021-02-23 15:03 ./usr/lib/phoenix/bin/argparse-1.4.0/argparse.py
-rwxr-xr-x root/root     32864 2021-02-23 15:03 ./usr/lib/phoenix/bin/daemon.py
-rwxr-xr-x root/root      1899 2021-02-23 15:03 ./usr/lib/phoenix/bin/end2endTest.py
-rw-r--r-- root/root      1621 2021-02-23 15:03 ./usr/lib/phoenix/bin/hadoop-metrics2-hbase.properties
-rw-r--r-- root/root      3056 2021-02-23 15:03 ./usr/lib/phoenix/bin/hadoop-metrics2-phoenix.properties
-rw-r--r-- root/root      1886 2021-02-23 15:03 ./usr/lib/phoenix/bin/hbase-omid-client-config.yml
-rw-r--r-- root/root      1084 2021-02-23 15:03 ./usr/lib/phoenix/bin/hbase-site.xml
-rw-r--r-- root/root      2583 2021-02-23 15:03 ./usr/lib/phoenix/bin/log4j.properties
-rwxr-xr-x root/root      5194 2021-02-23 15:03 ./usr/lib/phoenix/bin/performance.py
-rwxr-xr-x root/root      2804 2021-02-23 15:03 ./usr/lib/phoenix/bin/pherf-standalone.py
-rwxr-xr-x root/root      9037 2021-02-23 15:03 ./usr/lib/phoenix/bin/phoenix_utils.py
-rwxr-xr-x root/root      2814 2021-02-23 15:03 ./usr/lib/phoenix/bin/psql.py
-rw-r--r-- root/root      1823 2021-02-23 15:03 ./usr/lib/phoenix/bin/readme.txt
-rwxr-xr-x root/root      4646 2021-02-23 15:03 ./usr/lib/phoenix/bin/sqlline.py
-rwxr-xr-x root/root      7226 2021-02-23 15:03 ./usr/lib/phoenix/bin/traceserver.py
-rw-r--r-- root/root 152051939 2021-02-23 15:03 ./usr/lib/phoenix/phoenix-client-hbase-2.2-5.1.0.jar
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/share/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/share/doc/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/share/doc/phoenix/
-rw-r--r-- root/root     33895 2020-01-22 15:10 ./usr/share/doc/phoenix/LICENSE.gz
-rw-r--r-- root/root      3325 2020-01-22 15:10 ./usr/share/doc/phoenix/NOTICE.gz
drwxr-xr-x root/root         0 2020-01-22 15:10 ./usr/share/doc/phoenix/examples/
-rw-r--r-- root/root       149 2020-01-22 15:10 ./usr/share/doc/phoenix/examples/STOCK_SYMBOL.csv
-rw-r--r-- root/root       183 2020-01-22 15:10 ./usr/share/doc/phoenix/examples/STOCK_SYMBOL.sql
-rw-r--r-- root/root      2091 2020-01-22 15:10 ./usr/share/doc/phoenix/examples/WEB_STAT.csv
-rw-r--r-- root/root       296 2020-01-22 15:10 ./usr/share/doc/phoenix/examples/WEB_STAT.sql
-rw-r--r-- root/root       563 2020-01-22 15:10 ./usr/share/doc/phoenix/examples/WEB_STAT_QUERIES.sql
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/share/doc/phoenix-client/
-rw-r--r-- root/root       131 2021-02-23 15:03 ./usr/share/doc/phoenix-client/changelog.Debian.gz
-rw-r--r-- root/root     24257 2021-02-05 08:27 ./usr/share/doc/phoenix-client/changelog.gz
-rw-r--r-- root/root       318 2021-02-23 15:03 ./usr/share/doc/phoenix-client/copyright
lrwxrwxrwx root/root         0 2021-02-23 15:03 ./usr/lib/phoenix/phoenix-client.jar -> phoenix-client-hbase-2.2-5.1.0.jar
$ cd provisioner/docker
$ ./docker-hadoop.sh -d -k hdfs,hbase -L -C config_debian-10.yaml -c 3

...

$ for i in $(seq 1 3); do
>   if [ $i -eq 1 ]; then ./docker-hadoop.sh --exec $i apt-get install -y phoenix-client; fi
>   ./docker-hadoop.sh --exec $i apt-get install -y phoenix
>   ./docker-hadoop.sh --exec $i ln -s /usr/lib/phoenix/phoenix-server.jar /usr/lib/hbase/lib/
>   ./docker-hadoop.sh --exec $i systemctl restart hbase-regionserver
>   if [ $i -eq 1 ]; then ./docker-hadoop.sh --exec $i systemctl restart hbase-master; fi
> done

...

$ ./docker-hadoop.sh --exec 1 python3 /usr/lib/phoenix/bin/sqlline.py

...

Connected to: Phoenix (version 5.1)
Driver: PhoenixEmbeddedDriver (version 5.1)
Autocommit status: true
Transaction isolation: TRANSACTION_READ_COMMITTED
sqlline version 1.9.0
0: jdbc:phoenix:> CREATE TABLE t(c VARCHAR PRIMARY KEY);
No rows affected (2.476 seconds)
0: jdbc:phoenix:> UPSERT INTO t VALUES ('foo');
1 row affected (0.138 seconds)
0: jdbc:phoenix:> SELECT * FROM t;
+-----+
|  C  |
+-----+
| foo |
+-----+
1 row selected (0.072 seconds)

@iwasakims
Copy link
Member

iwasakims commented Feb 24, 2021

Thanks for working on this, @sekikn.

ln -s /usr/lib/phoenix/phoenix-server.jar /usr/lib/hbase/lib/

Should this symbolic link be contained in the phoenix package? Creating the symbolic link by scriptlet could be an another option.

[root@b0e6cfbecc83 /]# /usr/lib/phoenix/bin/sqlline.py
env: ‘python’: No such file or directory

I got this error on CentOS 8 bacause python2/python3 was not installed. The phoenix-client RPM should requires python3?

@iwasakims
Copy link
Member

iwasakims commented Feb 24, 2021 

$ rpm -qlp output/phoenix/noarch/phoenix-5.1.0-1.el7.noarch.rpm 
/etc/default/phoenix
/usr/lib/phoenix/phoenix-pherf-5.1.0.jar
/usr/lib/phoenix/phoenix-server-hbase-2.2-5.1.0.jar
/usr/lib/phoenix/phoenix-server.jar
  • Should this package be phoenix-server rather than phoenix?
  • There seems to be no init script using the /etc/default/phoenix.

@iwasakims
Copy link
Member

iwasakims commented Feb 24, 2021 

$ rpm -qlp output/phoenix/noarch/phoenix-client-5.1.0-1.el7.noarch.rpm 
/etc/phoenix/conf.dist
...
/usr/lib/phoenix/bin/traceserver.py
  • /etc/phoenix/conf.dist seems not to be used.
  • traceserver.py looks not client-side module. It will not work because relevant Java classes are not contained in phoenix-client.jar (nor phoenix-server.jar).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@sekikn @iwasakims