Skip to content

rbalamohan/tez-autobuild

Repository files navigation

tez-autobuild

To set up hive-tez on an HDP2 Sandbox VM, log in as root and in this repo, do

# make dist install

That should pull tez, hive-tez, build it and install it locally in ./dist/hive.

LLAP

Starting LLAP, requires you to have a working install of Apache Slider and a Zookeeper instance used by the YARN registry for co-ordination between nodes.

edit slider-gen.sh to fix your JAVA_HOME, pick your Xmx, container and cache sizing (basic rule = container size > (Xmx + cache)).

Run # ./slider-gen.sh

it generates a run.sh script in the local dir with the configuration for running (including date).

    ./llap-yarn-<date>/run.sh

Confirm application is running with

    yarn top

Alternatively, hive llap status tool can be used to check if all nodes are running llap daemons

    ./dist/hive/bin/hive --service llapstatus -w

If that fails to startup, check whether you have JDK8 in the right location specified in JAVA_HOME.

Finally, to test this out, you can do

# ./dist/hive/bin/beeline -u jdbc://localhost:<port>/
beeline> select count(1) from sample_07;

and it should use LLAP. Switch in and out with hive.llap.execution.mode, the execution will switch between container and llap.

alt tag

And for anything you want to override in local settings (like HIVE_CONF_DIR), create a file named local.mk and add the Makefile variables to that file.

About

A Tez dev-setup for HDP2 sandbox

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published