Skip to content

Commit

Permalink
Build RPM for presto server
Browse files Browse the repository at this point in the history 
Added the rpm build logic to a new module called presto-server-rpm.
We use the tar.gz built from presto-server module, untar it to
presto-server-rpm/target and then use it to build the rpm with rpm-maven-plugin

The rpm build is optional and will only be invoked using '-P rpmbuild'.
Run: 'mvn clean install -P rpmbuild' and rpm will be under
presto-server-rpm/target/rpm/presto/RPMS/x86_64

In post-install of rpm, we want to populate the node.properties file originally
templated with the node.id to have the actual value of uuidgen on the
node. This will enable us to have a single-node presto installed by just
running rpm -i.

In pre-install, we check if Java installed is Oracle's java version 1.8.
The install will fail if this vendor and version is not found. We did
not want to add jdk as the presto rpm pre-requisite, hence this check.

Also added init.d scripts to run start/stop/restart and status
on Presto server. We also provide chkconfig to init.d script.
  • Loading branch information
@martint
Anu Sudarsan authored and martint committed Jun 16, 2015
1 parent 3325ab0 commit 8fe9db9
Show file tree
Hide file tree
Showing 10 changed files with 426 additions and 0 deletions.
7 changes: 7 additions & 0 deletions pom.xml
View file
Expand Up @@ -87,6 +87,7 @@
<module>presto-cli</module>
<module>presto-benchmark-driver</module>
<module>presto-server</module>
<module>presto-server-rpm</module>
<module>presto-docs</module>
<module>presto-verifier</module>
</modules>
Expand Down Expand Up @@ -206,6 +207,12 @@
<version>${project.version}</version>
</dependency>

<dependency>
<groupId>com.facebook.presto</groupId>
<artifactId>presto-server-rpm</artifactId>
<version>${project.version}</version>
</dependency>

<dependency>
<groupId>com.facebook.presto</groupId>
<artifactId>presto-tests</artifactId>
Expand Down
50 changes: 50 additions & 0 deletions presto-server-rpm/README.md
View file
@@ -0,0 +1,50 @@
# Presto RPM

## RPM Package Build And Usage

You can now build an RPM package for Presto server and install Presto using the RPM. Thus, the installation will be easier to manage on RPM-based systems.

Building the RPM is currently an optional maven task and can be enabled by the 'rpmbuild' flag. From the locally cloned presto repository, run:

mvn install -P rpmbuild

This will create the RPM under the directory `presto-server-rpm/target/rpm/presto/RPMS/x86_64/`

The RPM has a pre-requisite of Python >= 2.4. It also needs Oracle Java 1.8 update 40 (8u40 64-bit) pre-installed. The RPM installation will fail if any of these requirements are not
satisfied.

To install Presto using an RPM, run:

rpm -i presto-server-<version>-1.0.x86_64.rpm

This will install Presto in single node mode, where both coordinator and workers are co-located on localhost. This will deploy the necessary default configurations along with a service script to control the Presto server process.

Uninstalling the RPM is like uninstalling any other RPM, just run:

rpm -e presto

Note: During uninstall, any Presto related files deployed will be deleted except for the Presto logs directory `/var/log/presto`.

## Control Scripts

The Presto RPM will also deploy service scripts to control the Presto server process. The script is configured with chkconfig,
so that the service is started automatically on OS boot. After installing Presto from the RPM, you can run:

service presto [start|stop|restart|status]

## Installation directory structure

We use the following directory structure to deploy various Presto artifacts.

* /usr/lib/presto/lib/: Various libraries needed to run the product. Plugins go in a plugin subdirectory.
* /etc/presto: General Presto configuration files like node.properties, jvm.config, config.properties. Connector configs go in a catalog subdirectory
* /etc/presto/env.sh: Java installation path used by Presto
* /var/log/presto: Log files
* /var/lib/presto/data: Data directory
* /usr/shared/doc/presto: Docs
* /etc/rc.d/init.d/presto: Control script

The node.properties file requires the following two additional properties since our directory structure is different from what standard Presto expects.

plugin.config-dir=/etc/presto/catalog
plugin.dir=/usr/lib/presto/lib/plugin
175 changes: 175 additions & 0 deletions presto-server-rpm/pom.xml
View file
@@ -0,0 +1,175 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>com.facebook.presto</groupId>
<artifactId>presto-root</artifactId>
<version>0.108-SNAPSHOT</version>
</parent>

<artifactId>presto-server-rpm</artifactId>
<name>presto-server-rpm</name>

<dependencies>
<dependency>
<groupId>com.facebook.presto</groupId>
<artifactId>presto-server</artifactId>
<version>${project.version}</version>
<type>tar.gz</type>
<scope>runtime</scope>
</dependency>
</dependencies>

<profiles>
<profile>
<id>rpmbuild</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<build>
<!-- Untar presto-server tgz to target build folder -->
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>unpack</id>
<goals>
<goal>unpack</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>com.facebook.presto</groupId>
<artifactId>presto-server</artifactId>
<version>${project.version}</version>
<type>tar.gz</type>
<outputDirectory>${project.build.directory}</outputDirectory>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
<!-- Build presto-server rpm using the untarred artifacts -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>rpm-maven-plugin</artifactId>
<version>2.1.2</version>
<executions>
<execution>
<id>generate-rpm</id>
<goals>
<goal>rpm</goal>
</goals>
</execution>
</executions>
<configuration>
<name>presto</name>
<version>${project.version}</version>
<release>1.0</release>
<group>Applications</group>
<description>Presto Server RPM Package.</description>
<needarch>x86_64</needarch>
<requires>
<require>python >= 2.4</require>
</requires>
<mappings>
<mapping>
<!-- This should go to /usr/bin eventually. But that needs modifying launcher.py in
airlift, to have a configurable option for install_path -->
<directory>/usr/lib/presto/bin</directory>
<!-- make sure laucher scripts are executable -->
<filemode>755</filemode>
<sources>
<source>
<location>${project.build.directory}/${server.tar.package}/bin
</location>
</source>
</sources>
</mapping>
<mapping>
<directory>/etc/rc.d/init.d</directory>
<!-- make sure init.d scripts are executable -->
<filemode>755</filemode>
<sources>
<source>
<location>src/main/resources/dist/etc/rc.d/init.d</location>
</source>
</sources>
</mapping>
<mapping>
<!-- This should go to just /usr/lib/presto eventually. But that needs modifying
launcher.py in airlift, to have a configurable option for install_path -->
<directory>/usr/lib/presto/lib</directory>
<sources>
<source>
<location>${project.build.directory}/${server.tar.package}/lib
</location>
</source>
</sources>
</mapping>
<mapping>
<directory>/usr/lib/presto/lib/plugin</directory>
<sources>
<source>
<location>
${project.build.directory}/${server.tar.package}/plugin
</location>
</source>
</sources>
</mapping>
<mapping>
<directory>/etc/presto</directory>
<sources>
<source>
<location>src/main/resources/dist/config</location>
</source>
</sources>
</mapping>
<mapping>
<directory>/usr/shared/doc/presto</directory>
<sources>
<source>
<location>${project.build.directory}/${server.tar.package}
</location>
<!-- Just include Presto's README for now.-->
<includes>
<include>README.txt</include>
</includes>
</source>
</sources>
</mapping>
<!-- Add these mappings so that .spec knows these dirs are to be removed too on rpm -e -->
<mapping>
<directory>/usr/lib/presto</directory>
</mapping>
<mapping>
<directory>/usr/lib/presto/lib</directory>
</mapping>
</mappings>
<preinstallScriptlet>
<scriptFile>src/main/resources/scripts/preinstall</scriptFile>
</preinstallScriptlet>
<postinstallScriptlet>
<scriptFile>src/main/resources/scripts/postinstall</scriptFile>
</postinstallScriptlet>
<postremoveScriptlet>
<scriptFile>src/main/resources/scripts/postremove</scriptFile>
</postremoveScriptlet>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>

<properties>
<air.main.basedir>${project.parent.basedir}</air.main.basedir>
<air.check.skip-checkstyle>true</air.check.skip-checkstyle>
<server.tar.package>presto-server-${project.version}</server.tar.package>
</properties>
</project>
7 changes: 7 additions & 0 deletions presto-server-rpm/src/main/resources/dist/config/config.properties
View file
@@ -0,0 +1,7 @@
#single node install config
coordinator=true
node-scheduler.include-coordinator=true
http-server.http.port=8080
task.max-memory=1GB
discovery-server.enabled=true
discovery.uri=http://localhost:8080
8 changes: 8 additions & 0 deletions presto-server-rpm/src/main/resources/dist/config/jvm.config
View file
@@ -0,0 +1,8 @@
-server
-Xmx16G
-XX:-UseBiasedLocking
-XX:+UseG1GC
-XX:+ExplicitGCInvokesConcurrent
-XX:+HeapDumpOnOutOfMemoryError
-XX:+UseGCOverheadLimit
-XX:OnOutOfMemoryError=kill -9 %p
5 changes: 5 additions & 0 deletions presto-server-rpm/src/main/resources/dist/config/node.properties
View file
@@ -0,0 +1,5 @@
node.environment=test
node.id=${uuid-generated-nodeid}
node.data-dir=/var/lib/presto/data
plugin.config-dir=/etc/presto/catalog
plugin.dir=/usr/lib/presto/lib/plugin
84 changes: 84 additions & 0 deletions presto-server-rpm/src/main/resources/dist/etc/rc.d/init.d/presto
View file
@@ -0,0 +1,84 @@
#!/bin/bash
#
# Manages a Presto node service
#
# chkconfig: 345 85 15
# description: Presto node
#
### BEGIN INIT INFO
# Provides: presto
# Short-Description: presto node
# Default-Start: 3 4 5
# Default-Stop: 0 1 2 6
# Required-Start: $syslog $remote_fs
# Required-Stop: $syslog $remote_fs
# Should-Start:
# Should-Stop:
### END INIT INFO

SCRIPT_NAME=$0
ACTION_NAME=$1
SERVICE_NAME='presto'

# Launcher Config.
# Use data-dir from node.properties file (assumes it to be at /etc/presto). For other args use defaults from rpm install
NODE_PROPERTIES=/etc/presto/node.properties
DATA_DIR=$(grep -Po "(?<=^node.data-dir=).*" $NODE_PROPERTIES)
CONFIGURATION=(--launcher-config /usr/lib/presto/bin/launcher.properties --data-dir "$DATA_DIR" --node-config "$NODE_PROPERTIES" --jvm-config /etc/presto/jvm.config --config /etc/presto/config.properties --launcher-log-file /var/log/presto/launcher.log --server-log-file /var/log/presto/server.log)

source /etc/presto/env.sh

start () {
echo "Starting ${SERVICE_NAME} "
if [ -z "$JAVA8_HOME" ]
then
echo "Warning: No value found for JAVA8_HOME. Default Java will be used."
/usr/lib/presto/bin/launcher start ${CONFIGURATION}
else
PATH=${JAVA8_HOME}/bin:$PATH /usr/lib/presto/bin/launcher start "${CONFIGURATION[@]}"
fi
return $?
}

stop () {
echo "Stopping ${SERVICE_NAME} "
/usr/lib/presto/bin/launcher stop "${CONFIGURATION[@]}"
return $?
}

status () {
echo "Getting status for ${SERVICE_NAME} "
/usr/lib/presto/bin/launcher status "${CONFIGURATION[@]}"
return $?
}


restart () {
/usr/lib/presto/bin/launcher restart "${CONFIGURATION[@]}"
return $?
}

# TODO: Add kill

usage () {
echo $"Usage: ${SCRIPT_NAME} {start|stop|restart|status}"
return 3
}

case "${ACTION_NAME}" in
start)
start
;;
stop)
stop
;;
restart)
restart
;;
status)
status
;;
*)
echo $"Usage: $0 {start|stop|restart|status}"
exit 3
esac
14 changes: 14 additions & 0 deletions presto-server-rpm/src/main/resources/scripts/postinstall
View file
@@ -0,0 +1,14 @@
# Post installation script

# Dynamically populated directories that we expect to exist but do
# not want to remove when removing the RPM
install --directory --mode=775 /var/log/presto

# Populate node.id from uuidgen by replacing template with the node uuid
sed -i "s/\${uuid-generated-nodeid}/$(uuidgen)/g" /etc/presto/node.properties

# move the presto_env.sh created during pre-install to presto config location
if [ -e /tmp/presto_env.sh ]
then
mv /tmp/presto_env.sh /etc/presto/env.sh
fi
9 changes: 9 additions & 0 deletions presto-server-rpm/src/main/resources/scripts/postremove
View file
@@ -0,0 +1,9 @@
# Post erase script

# Delete the conf directory manually during uninstall.
# rpm -e wont remove it, because this directory was manually updated in postinstall
rm -rf /etc/presto
# Delete the data directory manually during uninstall.
# rpm -e wont remove it, because this directory may later contain files not
# deployed by the rpm
rm -rf /var/lib/presto

0 comments on commit 8fe9db9

Please sign in to comment.