Skip to content
Scripts for building Cloudera Manager parcel and CSD for Livy Spark Server
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

Livy Spark Server parcel and CSD

Livy is a wrapper around spark-submit to expose Spark as a REST service for integration with external applications.

As it is still in beta, it is not included in the CDH bundle (as of CDH 5.5.1). This scripts in this repo creates both a Livy parcel and CSD for deploying it with Cloudera Manager.


Cloudera publishes a tool for making CM parcels. Install it on your workstation like so:

  1. mkdir -p ~/github/cloudera; cd ~/github/cloudera
  2. git clone; cd cm_ext
  3. mvn package



  1. Create the Livy parcel: ./ <Version> <Distro>. Example: ./ 1.0 wheezy
  2. Serve the parcel: ./
  3. In CM, add your machine to the list of Remote Parcel Repository URLs and click Check for New Parcels.
  4. Download, Distribute, Activate. No need to restart the cluster as this parcel is not a dependency for any service.


  1. Create the CSD: ./ <Version>. Example: ./ 1.0. The CSD version is independent of the parcel version.
  2. Copy the CSD jar (LIVY-1.0.jar) to CM host at /opt/cloudera/csd/ and change the ownership of the jar to cloudera-scm:cloudera-scm
  3. Restart CM /etc/init.d/cloudera-scm-server restart
  4. Restart Cloudera Management Service from the CM UI.
  5. Install the Livy service with Add a service option in CM.

Note: Step 3 and 4 can be avoided by using the experimental api for installing CSDs without restarting CM with a GET https://CM:7180/cmf/csd/install?csdName=LIVY-1.0

Managing Livy

Livy has been defined as a javabased service in CSD which enables the basic monitoring and configuration relevant to java process through CM. Additional Livy parameters has also be added to the CSD to enable configuration management from CM. We are forcing logback as the logging implementation.

As of now, custom CSDs support declaring custom montitoring metrics but no way to publish them from the service.

You can’t perform that action at this time.