HAWQ-326. Support RPM package for Apache HAWQ.

[radarwave]

@edespino @kdunn926 @paul-guo- @huor
Please help to review the RPM spec file for HAWQ.

This spec file comes from a history file which contributed by Lei, Ed and me. We also added some comments from others like Kyle.

Later we can add an option 'make rpm' to make the user create the rpm package easier.

[radarwave]

@xunzhang I guess you are doing some package works on many systems.

[xunzhang]

@radarwave TIL, thanks!

[edespino]

As a general spec file to produce a pre-built binary releaes of HAWQ, this is a good starting point. Ideally, the spec file will also have a "build" section which is used to build the release from a source tarball. You can view the Apache Bigtop repository (https://github.com/apache/bigtop/tree/master/bigtop-packages/src/rpm) to see references to spec files which will build their respective products from source.

Here are two more helpful references to spec files containing build sections:

• https://github.com/hurdad/impala-rpm/blob/master/impala.spec
• https://github.com/apache/httpd/blob/trunk/build/rpm/httpd.spec.in

[edespino]

It would be better to provide a real description than rather using the summary. How about something like:

Apache HAWQ (incubating) combines exceptional MPP-based analytics
performance, robust ANSI SQL compliance, Hadoop ecosystem
integration and manageability, and flexible data-store format
support, all natively in Hadoop, no connectors required.

Built from a decade’s worth of massively parallel
processing (MPP) expertise developed through the creation of open
source Greenplum® Database and PostgreSQL, HAWQ enables you to
swiftly and interactively query Hadoop data, natively via HDFS.

[radarwave]

Make sense, done.

[radarwave]

@edespino Good point to build HAWQ rpm from source tarball. We should be able to build a rpm package by both pre-build binary tarball and source tarball.

[edespino]

@radarwave - to keep things moving along for the upcoming Apache HAWQ 2.2.0.0-incubating release, I suggest you open a Jira to address building from source tarball in the spec file and target it for the next release. This may be a very time-consuming process to get it right.

[edespino]

@radarwave - As soon as this is committed, this should be the only spec file used to generate HAWQ rpms for both Open Source or commercial releases. Is this a true statement?

[radarwave]

@edespino A jira is created to trace build rpm package from source tarball, this time we will focus on rpm build from binary tarball.

See:
https://issues.apache.org/jira/browse/HAWQ-1392

[radarwave]

@edespino I think this is only for open source release.

[paul-guo-]

If I understand correctly, we seem to make things a bit complicated. Why not generate a src.rpm and rpm based on source tarball + one spec file directly?

Below is an example from my repo:
https://github.com/paul-guo-/moxi

srpm: $(TARBALL) $(SPEC_FILE)
rpmbuild --define '_sourcedir $(SRCDIR)' --define '_srcrpmdir $(DESTDIR)' -bs $(SPEC_FILE)

$(TARBALL): moxi-1.8.0_8_g52a5fa8
tar zcf $@ $^

build: srpm
rpmbuild --rebuild moxi-*.src.rpm

[kdunn926]

+1 to @paul-guo- 's approach. One spec file to rule them all. Will be more maintainable.

[radarwave]

To build hawq rpm package from source tarball, we need to fix below issues:

1. 'make install DESTDIR=/destdir/usr/local/apache-hawq' is not working for all the files. For example: our configuration files under 'etc' folder. Build rpm from source tarball relies on this 'DESTDIR' option.

2. Some python modules(pygresql etc...) compile is not in 'make' stage, but in 'make install' stage, it will cause compile errors if we enabled 'DESTDIR' as it can't find some pre-installed dependencies in the '--prefix' directory. See:

Traceback (most recent call last): File "setup.py", line 104, in <module> mk_include() File "setup.py", line 75, in mk_include for f in os.listdir(pg_include_dir_server): OSError: [Errno 2] No such file or directory: '/usr/local/apache-hawq/include/postgresql/server'

Will spend some more time to check if we can fix above issues quickly.

Now we have a hack to set the install directory directly to %{buildroot}%{installdir}, then we do 'make install' without 'DESTDIR' option. This way requires to do below hack to skip a rpmbuild check:

 in ~/.rpmmacros, add the line:
%__arch_install_post   /usr/lib/rpm/check-rpaths   /usr/lib/rpm/check-buildroot

[radarwave]

This PR is for build HAWQ rpm from binary tarball, since now we chose using source tarball to build rpm packages, so open another PR for the review.

New PR link is:
#1195

Will close this PR if no further comments. Thanks.