Skip to content

Commit

Permalink
Merge pull request #317 from VictorLowther/feature/sledgehammer/opens…
Browse files Browse the repository at this point in the history
…tack-os-build

Update Crowbar to generate Centos 6.2 sledgehammer images. [5/6]
  • Loading branch information
VictorLowther committed Mar 22, 2012
2 parents 243dad1 + 00b4ef3 commit 568cf16
Show file tree
Hide file tree
Showing 21 changed files with 559 additions and 55 deletions.
12 changes: 1 addition & 11 deletions README.build
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,6 @@ Prerequisites:
If you want to allow build_crowbar to run the above commands as root
without having to enter a password each time, the build_crowbar.sh
includes a sample line you can fix up and add to /etc/sudoers.
* A working Sledgehammer tftpboot image.
Sledgehammer is a minimal CentOS 5.6 pxe bootable image that we use
to perform hardware discovery and configuration for devices that do not
have low-level management tools for Ubuntu. You can find information on
how to build Sledgehamemr by reading its HOWTO.Non.Redhat file. We do
not do fully automated builds of Sledgehammer due to a lack of a CentOS
equivalent of debootstrap.

Command Line Parameters:
* The first command line parameter is the OS you want to
Expand Down Expand Up @@ -146,9 +139,6 @@ Customization:
* CROWBAR_DIR
The directory that the Crowbar source is cheched out to.
Defaults to the directory that build_crowbar.sh is in.
* SLEDGEHAMMER_DIR
The directory that the Sledgehammer source is checked out to.
Defaults to $CROWBAR_DIR/../sledgehammer
* VCS_CLEAN_CMD
This is the command that build_crowbar.sh will run to clean the tree before
staging the Crowbar build.
Expand Down Expand Up @@ -190,7 +180,7 @@ When build_crowbar.sh is invoked, it performs the following processes in order:
This metadata will drive the rest of the install -- we need it to
figure out dependency relations between barclamps, what packages and files
to stage, and how to invoke any external build processes we might need.
14: Make sure we have a Sledgehammer image handy.
14: Make sure we have a Sledgehammer image handy, and build it if we don't.
15: If we don't have the OS ISO to stage on, and we know how to get one, then
download the .ISO we will need.
16: Clean out any leftovers from the last build, and make sure that
Expand Down
16 changes: 14 additions & 2 deletions README.sledgehammer
Original file line number Diff line number Diff line change
@@ -1,2 +1,14 @@
In order to build Crowbar, you need to build Sledgehammer first.
Sledgehammer can be found at github:dellcloudedge/crowbar-sledgehammer
Sledgehammer is the component of Crowbar that we use to perform
initial discovery and configuration of systems with. It is a slightly
modified Centos 6.2 live environment. To build it, you need:

* A CentOS 6.2 install DVD from bittorrent or your favorite CentOS
mirror. This install DVD will need to go in $ISO_LIBRARY
(usually $HOME/.crowbar-build-cache/iso).
* Ruby, rpm, and rpm2cpio.

By default, the Crowbar build process will try and build Sledgehammer
if it is not already present in the build cache, so you should not
need to do anything special to build Sledgehammer. If you need to
update Sledgehammer, run the build_sledgehammer.sh script in the main
Crowbar checkout.
1 change: 0 additions & 1 deletion README.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ This wiki maintains information about the operation of Crowbar. Documentation fo
Please consult the release notes for known issues

To build your own copy of Crowbar, consult the readme (https://github.com/dellcloudedge/crowbar/blob/master/README.build) in the crowbar repos. Some important notes about the build process:
* Build requires a component known as "sledgehammer" that is a CentOS based image for discovery. This TFTPboot image does not change often and is not part of the normal build steps. See the DellCloudEdge/crowbar-sledgehammer repo for details.
* Build process has been tested on Ubuntu 10.10 and CentOS

With that said, from a fresh install off the crowbar-dev.iso, the next steps are:
Expand Down
2 changes: 1 addition & 1 deletion barclamps/crowbar
25 changes: 5 additions & 20 deletions build_crowbar.sh
Original file line number Diff line number Diff line change
Expand Up @@ -96,10 +96,6 @@ fi
die "$CROWBAR_DIR is not a git checkout of Crowbar!"
export CROWBAR_DIR

# Location of the Sledgehammer source tree. Only used if we cannot
# find Sledgehammer in $SLEDGEHAMMER_PXE_DIR above.
[[ $SLEDGEHAMMER_DIR ]] || SLEDGEHAMMER_DIR="${CROWBAR_DIR}/../crowbar-sledgehammer"

# Command to run to clean out the tree before starting the build.
# By default we want to be relatively pristine.
[[ $VCS_CLEAN_CMD ]] || VCS_CLEAN_CMD='git clean -f -d'
Expand Down Expand Up @@ -399,11 +395,11 @@ do_crowbar_build() {

debug "Checking for Sledgehammer."
# Make sure Sledgehammer has already been built and pre-staged.
if ! [[ -f $SLEDGEHAMMER_DIR/bin/sledgehammer-tftpboot.tar.gz || \
-f $SLEDGEHAMMER_PXE_DIR/initrd0.img ]]; then
echo "Slegehammer TFTP image missing!"
echo "Please build Sledgehammer from $SLEDGEHAMMER_DIR before building Crowbar."
exit 1
if ! [[ -f $SLEDGEHAMMER_PXE_DIR/initrd0.img ]]; then
debug "Slegehammer TFTP image missing!"
debug "Attempting to build Sledgehammer:"
"$CROWBAR_DIR/build_sledgehammer.sh" || \
die "Unable to build Sledgehammer. Cannot build Crowbar."
fi

# Fetch the OS ISO if we need to.
Expand Down Expand Up @@ -517,17 +513,6 @@ do_crowbar_build() {

# Copy over the bits that Sledgehammer will look for.
debug "Copying over Sledgehammer bits"
# If we need to copy over a new Sledgehammer image, do so.
if [[ $SLEDGEHAMMER_DIR/bin/sledgehammer-tftpboot.tar.gz -nt \
$SLEDGEHAMMER_PXE_DIR/initrd0.img ]]; then
( cd $SLEDGEHAMMER_PXE_DIR
debug "Extracting new Sledgehammer TFTP boot image"
rm -rf .
cd ..
tar xzf "$SLEDGEHAMMER_DIR/bin/sledgehammer-tftpboot.tar.gz"
rm -f "$SLEDGEHAMMER_DIR/bin/sledgehammer-tftpboot.tar.gz"
)
fi
cp -a "$SLEDGEHAMMER_PXE_DIR"/* "$BUILD_DIR/discovery"

# Make our image
Expand Down
1 change: 1 addition & 0 deletions build_lib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -362,6 +362,7 @@ make_chroot() {
sudo mkdir -p "$CHROOT/$CHROOT_PKGDIR"
sudo mkdir -p "$CHROOT/$CHROOT_GEMDIR"
__make_chroot
in_chroot ln -s /proc/self/mounts /etc/mtab

if [[ $ALLOW_CACHE_UPDATE = true ]]; then
read_base_repos
Expand Down
168 changes: 168 additions & 0 deletions build_sledgehammer.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
#!/bin/bash
#
# Build a sledgehammer image for Crowbar and put it in the build cache.

# Copyright 2011, Dell
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Author: VictorLowther

# We always use the C language and locale
export LANG="C"
export LC_ALL="C"

GEM_RE='([^0-9].*)-([0-9].*)'

readonly currdir="$PWD"
export PATH="$PATH:/sbin:/usr/sbin:/usr/local/sbin"

if ! which cpio &>/dev/null; then
die "Cannot find cpio, we cannot proceed."
fi

if ! which rpm rpm2cpio &>/dev/null; then
die "Cannot find rpm and rpm2cpio, we cannot proceed."
fi

if ! which ruby &>/dev/null; then
die "You must have Ruby installed to run this script. We cannot proceed."
fi


# Source our config file if we have one
[[ -f $HOME/.build-crowbar.conf ]] && \
. "$HOME/.build-crowbar.conf"

# Look for a local one.
[[ -f build-crowbar.conf ]] && \
. "build-crowbar.conf"

# Set up our proxies if we were asked to.
if [[ $USE_PROXY = "1" && $PROXY_HOST ]]; then
proxy_str="http://"
if [[ $PROXY_PASSWORD && $PROXY_USER ]]; then
proxy_str+="$PROXY_USER:$PROXY_PASSWORD@"
elif [[ $PROXY_USER ]]; then
proxy_str+="$PROXY_USER@"
fi
proxy_str+="$PROXY_HOST"
[[ $PROXY_PORT ]] && proxy_str+=":$PROXY_PORT"
[[ $no_proxy ]] || no_proxy="localhost,localhost.localdomain,127.0.0.0/8,$PROXY_HOST"
[[ $http_proxy ]] || http_proxy="$proxy_str/"
[[ $https_proxy ]] || https_proxy="$http_proxy"
export no_proxy http_proxy https_proxy
else
unset no_proxy http_proxy https_proxy
fi

# Always run in verbose mode for now.
VERBOSE=true

# OS to stage Sledgehammer on to. Defaults to CentOS 6.2
[[ $SLEDGEHAMMER_OS ]] || SLEDGEHAMMER_OS="centos-6.2"
OS_TO_STAGE="$SLEDGEHAMMER_OS"
OS_TOKEN="$OS_TO_STAGE"

# Location for caches that should not be erased between runs
[[ $CACHE_DIR ]] || CACHE_DIR="$HOME/.crowbar-build-cache"

# The directory that we will mount the OS .ISO on .
[[ $IMAGE_DIR ]] || \
IMAGE_DIR="$CACHE_DIR/$OS_TOKEN/sledgehammer-image"

# Location to store .iso images that we use in the build process.
# These are usually OS install DVDs that we will stage Crowbar on to.
[[ $ISO_LIBRARY ]] || ISO_LIBRARY="$CACHE_DIR/iso"

[[ $CHROOT ]] || CHROOT="$CACHE_DIR/$OS_TOKEN/sledgehammer-chroot"
sudo rm -rf "$CHROOT"

mkdir -p "$CACHE_DIR" "$IMAGE_DIR" "$CHROOT"

# Location of the Crowbar checkout we are building from.
[[ $CROWBAR_DIR ]] || CROWBAR_DIR="${0%/*}"
[[ $CROWBAR_DIR = /* ]] || CROWBAR_DIR="$currdir/$CROWBAR_DIR"
[[ -f $CROWBAR_DIR/build_crowbar.sh && -d $CROWBAR_DIR/.git ]] || \
die "$CROWBAR_DIR is not a git checkout of Crowbar!"
export CROWBAR_DIR

# Directory that holds our Sledgehammer PXE tree.
[[ $SLEDGEHAMMER_PXE_DIR ]] || SLEDGEHAMMER_PXE_DIR="$CACHE_DIR/tftpboot"

unset CROWBAR_BUILD_PID
# Source our common build functions
. "$CROWBAR_DIR/build_lib.sh" || exit 1
. "$CROWBAR_DIR/test_lib.sh" || exit 1

# Make sure that we actually know how to build the ISO we were asked to
# build. If we do not, print a helpful error message.
if ! [[ $OS_TO_STAGE && -d $CROWBAR_DIR/$OS_TO_STAGE-extra && \
-f $CROWBAR_DIR/$OS_TO_STAGE-extra/build_lib.sh ]]; then
cat <<EOF
You must pass the name of the operating system you want to stage Sledgehammer
on to. Valid choices are:
EOF
cd "$CROWBAR_DIR"
for d in *-extra; do
[[ -d $d && -f $d/build_lib.sh ]] || continue
echo " ${d%-extra}"
done
exit 1
fi

SLEDGEHAMMER_CHROOT_CACHE="$CACHE_DIR/sledgehammer/$OS_TO_STAGE/chroot_cache"
SLEDGEHAMMER_LIVECD_CACHE="$CACHE_DIR/sledgehammer/$OS_TO_STAGE/livecd_cache"

[[ -f $CROWBAR_DIR/$OS_TO_STAGE-extra/build_sledgehammer_lib.sh && \
-f $CROWBAR_DIR/$OS_TO_STAGE-extra/sledgehammer.ks ]] || \
die "Do not know how to build Sledgehammer on this OS!"

. "$CROWBAR_DIR/$OS_TO_STAGE-extra/build_lib.sh"

. "$CROWBAR_DIR/$OS_TO_STAGE-extra/build_sledgehammer_lib.sh"

debug "Mounting $ISO"
sudo mount -t iso9660 -o loop "$ISO_LIBRARY/$ISO" "$IMAGE_DIR" || \
die "Could not mount $ISO"

make_chroot
sudo cp "$CROWBAR_DIR/$OS_TO_STAGE-extra/sledgehammer.ks" "$CHROOT/mnt"
sudo cp "$CROWBAR_DIR/sledgehammer-common/"* "$CHROOT/mnt"
mkdir -p "$SLEDGEHAMMER_CHROOT_CACHE"
mkdir -p "$SLEDGEHAMMER_LIVECD_CACHE"
in_chroot mkdir -p /mnt/cache
sudo mount --bind "$SLEDGEHAMMER_CHROOT_CACHE" "$CHROOT/$CHROOT_PKGDIR"
sudo mount --bind "$SLEDGEHAMMER_LIVECD_CACHE" "$CHROOT/mnt/cache"
setup_sledgehammer_chroot
chroot_install livecd-tools tar
in_chroot touch /mnt/make_sledgehammer
in_chroot chmod 777 /mnt/make_sledgehammer
echo '#!/bin/bash' >>/mnt/make_sledgehammer
if [[ $USE_PROXY = "1" ]]; then
printf "\nno_proxy=%q\nhttp_proxy=%q\nexport no_proxy http_proxy\n" \
"$no_proxy" "$http_proxy" >> "$CHROOT/mnt/make_sledgehammer"
fi
cat >> "$CHROOT/mnt/make_sledgehammer" <<EOF
set -e
cd /mnt
livecd-creator --config=sledgehammer.ks --cache=./cache -f sledgehammer
rm -fr /mnt/tftpboot
livecd-iso-to-pxeboot sledgehammer.iso
/bin/rm /mnt/sledgehammer.iso
EOF
in_chroot /mnt/make_sledgehammer
cp -a "$CHROOT/mnt/tftpboot" "$CACHE_DIR/"
in_chroot /bin/rm -rf /mnt/tftpboot

[[ -f $CACHE_DIR/tftpboot/initrd0.img ]]
24 changes: 24 additions & 0 deletions centos-6.2-extra/build_sledgehammer_lib.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/bin/bash

# This file contains library routines needed to build Sledgehammer

EXTRA_REPOS=('http://mirror.centos.org/centos/6/os/$basearch' \
'http://mirror.centos.org/centos/6/updates/$basearch' \
'http://mirror.centos.org/centos/6/extras/$basearch' \
'http://mirror.pnl.gov/epel/6/$basearch' \
'http://www.nanotechnologies.qc.ca/propos/linux/centos-live/$basearch/live' \
'http://rbel.frameos.org/stable/el6/$basearch')

setup_sledgehammer_chroot() {
local repo rnum
sudo rm -f "$CHROOT/etc/yum.repos.d/"*
rnum=0
for repo in "${EXTRA_REPOS[@]}"; do
add_repos "bare r${rnum} 10 $repo"
rnum=$((rnum + 1))
done
# Make sure yum does not throw away our caches for any reason.
in_chroot /bin/sed -i -e '/keepcache/ s/0/1/' /etc/yum.conf
in_chroot sh -c "echo 'exclude = *.i386' >>/etc/yum.conf"
# fourth, have yum bootstrap everything else into usefulness
}
Loading

0 comments on commit 568cf16

Please sign in to comment.