Skip to content
Permalink
Browse files
BIGTOP-3774: Drop Docker Sandbox and Vagrant Provisioner (#978)
* BIGTOP-3774: Drop Docker Sandbox and Vagrant Provisioner

* BIGTOP-3774: address yuqi's comment
  • Loading branch information
evans-ye committed Aug 10, 2022
1 parent a528192 commit 659fa1c591d9f67a24617950cf9169dbf2863a7f
Show file tree
Hide file tree
Showing 23 changed files with 5 additions and 981 deletions.
@@ -24,6 +24,5 @@ sqoop: sean mackrory, youngwoo kim
tajo: yeongeon kim
test-artifacts and test-execution repos:
tez: evans ye, oflebbe
vagrant recipes: jay vyas, evans ye
ycsb: youngwoo kim
zookeeper: sean mackrory, rvs
@@ -26,7 +26,7 @@ The primary goal of Apache Bigtop is to build a community around the packaging a
Immediately Get Started with Deployment and Smoke Testing of BigTop
===================================================================

The simplest way to get a feel for how bigtop works, is to just cd into `provisioner` and try out the recipes under vagrant or docker. Each one rapidly spins up, and runs the bigtop smoke tests on, a local bigtop based big data distribution. Once you get the gist, you can hack around with the recipes to learn how the puppet/rpm/smoke-tests all work together, going deeper into the components you are interested in as described below.
The simplest way to get a feel for how bigtop works, is to just cd into `provisioner` and try out the docker recipes. It rapidly spins up, and runs the bigtop smoke tests on, a local bigtop based big data distribution. Once you get the gist, you can hack around with the recipes to learn how the puppet/rpm/smoke-tests all work together, going deeper into the components you are interested in as described below.

Quick overview of source code directories
=========================================
@@ -38,7 +38,7 @@ Quick overview of source code directories
* __test-artifacts__ : source for tests.
* __test-execution__ : maven pom drivers for running the integration tests found in test-artifacts.
* __bigtop-toolchain__ : puppet scripts for setting up an instance which can build Apache Bigtop, sets up utils like jdk/maven/protobufs/...
* __provisioner__ : Vagrant and Docker Provisioner that automatically spin up Hadoop environment with one click.
* __provisioner__ : Docker Provisioner that automatically spin up Hadoop environment with one click.
* __docker__ : Dockerfiles and Docker Sandbox build scripts.

Also, there is a new project underway, Apache Bigtop blueprints, which aims to create templates/examples that demonstrate/compare various Apache Hadoop ecosystem components with one another.
@@ -22,7 +22,7 @@ When running puppet apply, note that puppet's confdir is *underneath* the `--con
For example: If you have `site.csv` in `/etc/puppet/config`,
Then you should use `--confdir=/etc/puppet` , and puppet finds the config dir underneath.

As an end to end example, you can follow the `vagrant-puppet-vm` recipes to see how to set up
As an end to end example, you can follow the `provisioner/docker` recipes to see how to set up
a puppet managed bigtop hadoop installation. Those examples are gauranteed to work and
serve as a pedagogical round trip to the way bigtop integrates packaging, deployment, and
testing all into one package.
@@ -69,7 +69,6 @@
["/user","755","HCFS_SUPER_USER",null],
["/user/history","755","mapred","mapred"],
["/user/jenkins","777","jenkins",null],
["/user/vagrant","777","vagrant",null],
["/user/hive","777",null,null],
["/user/root","777","root",null],
["/user/hue","777","hue","hue"],
@@ -69,15 +69,6 @@ standalone with a command such as:

where <path_to_bigtop> is the cloned git repo.

## Installation of Tools for Bigtop Deployment

This is a separated set of manifests that helps to setup tools for Bigtop deployment.
The usage is as below:

puppet apply --modulepath=<path_to_bigtop> -e "include bigtop_toolchain::deployment_tools"

By applying the snippet, Vagrant will be installed(the Docker installation will be added soon).

## Optional development tools

This isn't a part of fundamental toolchain recipes as we are trying to contain the size of CI and dev-

This file was deleted.

This file was deleted.

This file was deleted.

0 comments on commit 659fa1c

Please sign in to comment.