BIGTOP-3568: Fix Ambari-2.7.5 RPM installation issues

[guyuqi]

Find correct python version path to create soft links for
resource_management and jinja.

[iwasakims]

This looks unstable if multiple python packages are installed. I think it is not so rare on CentOS 7/Cent OS 8.

# find /usr/lib -maxdepth 1 -name 'python*'
/usr/lib/python2.7
/usr/lib/python3.6

[guyuqi]

Yes，there may be multiple python-lib existed in /usr/lib.
The result of find /usr/lib -maxdepth 1 -name 'python*' is always sorted as python2.x, python3.x, etc.
And PYLIB_DIR=echo ${PYPATH} | awk '{print $1}'`` will always select first installed one (python2.7, ) for creating synbolic links.

Python binary (/usr/bin/python) is always link to the first installed python version:
/usr/bin/python -> python2

And in some envrionment ( like our puppet docker image：bigtop/puppet:trunk-centos-8), there is no python binary installed, but python lib directory is available:

whereis python
python: /usr/lib/python3.6 /usr/lib64/python3.6 /usr/include/python3.6m /usr/share/man/man1/python.1.gz

We could not get python version by the bash command python -V.
So I directly find the first installed python-lib directory to create synbolic links for resource_management and jinja.
IMO, it's the balanced approach among various environments, or do you have other good suggestions? Thanks.

[iwasakims]

The result of find /usr/lib -maxdepth 1 -name 'python*' is always sorted as python2.x, python3.x, etc.

Hmm. This did not stand on one of my CentOS environment.

$ find /usr/lib -maxdepth 1 -name 'python*'
/usr/lib/python3.6
/usr/lib/python2.7
/usr/lib/python2.6

Should we have explicit sort?

$ find /usr/lib -maxdepth 1 -name 'python*' | sort
/usr/lib/python2.6
/usr/lib/python2.7
/usr/lib/python3.6

[guyuqi]

Yes, thanks.

[iwasakims]

I got 2 issues on (re)installing ambari-server on CentOS 7.

• ‘/var/lib/ambari-server/resources/views/*.jar’: No such file or directory
• synbolic link created in the 1st pass seems to be staying.

# yum remove ambari-server
# yum install ambari-server
...
cp: cannot stat ‘/var/lib/ambari-server/resources/views/*.jar’: No such file or directory
  Installing : ambari-server-2.7.5.0-1.el7.noarch                                                       1/1 
/var/tmp/rpm-tmp.zHadY6: line 18: /usr/lib/python2.7/site-packages/resource_management: Is a directory
ln: failed to create symbolic link ‘./resource_management’: File exists
...

[guyuqi]

I got 2 issues on (re)installing ambari-server on CentOS 7.

• ‘/var/lib/ambari-server/resources/views/*.jar’: No such file or directory
• synbolic link created in the 1st pass seems to be staying.

# yum remove ambari-server
# yum install ambari-server
...
cp: cannot stat ‘/var/lib/ambari-server/resources/views/*.jar’: No such file or directory
  Installing : ambari-server-2.7.5.0-1.el7.noarch                                                       1/1 
/var/tmp/rpm-tmp.zHadY6: line 18: /usr/lib/python2.7/site-packages/resource_management: Is a directory
ln: failed to create symbolic link ‘./resource_management’: File exists
...

In rpm spec:

# remove RESOURCE_MANAGEMENT_DIR if it's a directory
if [ -d "$RESOURCE_MANAGEMENT_DIR" ]; then  # resource_management dir exists
  if [ ! -L "$RESOURCE_MANAGEMENT_DIR" ]; then # resource_management dir is not link
    rm -rf "$RESOURCE_MANAGEMENT_DIR"
  fi
fi

These links will be removed when install rpm installation.

And no issues occurred by directly using rpm -i :

rpm -ivh ambari-server-2.7.5.0-1.el7.noarch.rpm
rpm -qa | grep 'ambari'
rpm -e ambari-server-2.7.5.0-1.el7.noarch

[iwasakims]

These links will be removed when install rpm installation.

The RESOURCE_MANAGEMENT_DIR is symbolic link (created by the RPM) and left after uninstall. It causes error on reinstall. You can reproduce the issue by docker provisioner.

[root@f7ab394c86d1 /]# PYPATH=`find /usr/lib -maxdepth 1 -name 'python*'`
[root@f7ab394c86d1 /]# ls -l $PYPATH/site-packages/resource_management
lrwxrwxrwx 1 root root 45 Jul 10 12:36 /usr/lib/python2.7/site-packages/resource_management -> /usr/lib/ambari-agent/lib/resource_management

[root@f7ab394c86d1 /]# rpm -e ambari-server ambari-agent
cp: cannot stat ‘/etc/ambari-agent/conf’: No such file or directory

Note: This output shows SysV services only and does not include native
      systemd services. SysV configuration data might be overridden by native
      systemd configuration.

      If you want to list systemd services use 'systemctl list-unit-files'.
      To see services enabled on particular target use
      'systemctl list-dependencies [target]'.

ambari-server   0:off   1:off   2:on    3:on    4:on    5:on    6:off
warning: file /etc/ambari-agent/conf/logging.conf.sample: remove failed: No such file or directory
warning: file /etc/ambari-agent/conf/ambari-agent.ini: remove failed: No such file or directory
cp: cannot stat ‘/etc/ambari-server/conf’: No such file or directory

Note: This output shows SysV services only and does not include native
      systemd services. SysV configuration data might be overridden by native
      systemd configuration.

      If you want to list systemd services use 'systemctl list-unit-files'.
      To see services enabled on particular target use
      'systemctl list-dependencies [target]'.

warning: file /var/lib/ambari-server/resources/stacks/Bigtop/2.1: remove failed: No such file or directory
warning: file /var/lib/ambari-server/resources/stacks/Bigtop: remove failed: No such file or directory
warning: file /etc/ambari-server/conf/metrics.properties: remove failed: No such file or directory
warning: file /etc/ambari-server/conf/krb5JAASLogin.conf: remove failed: No such file or directory
warning: file /etc/ambari-server/conf: remove failed: No such file or directory

[root@f7ab394c86d1 /]# rpm -qa | grep ambari
[root@f7ab394c86d1 /]# ls -l $PYPATH/site-packages/resource_management
lrwxrwxrwx 1 root root 45 Jul 10 12:36 /usr/lib/python2.7/site-packages/resource_management -> /usr/lib/ambari-agent/lib/resource_management

[iwasakims]

Installing ambari-server created /resource_management linked to RESOURCE_MANAGEMENT_DIR. This looks unintentional.

[root@f7ab394c86d1 /]# ls -lh /resource_management
lrwxrwxrwx 1 root root 46 Jul 10 12:52 /resource_management -> /usr/lib/ambari-server/lib/resource_management

[iwasakims]

anyway the original issue reported on BIGTOP-3568 was addressed. You can address issues on uninstall/reinstall in follow-up JIRA.

[guyuqi]

Thanks, @iwasakims