Permalink
Browse files

initial checkin

  • Loading branch information...
0 parents commit 0a97a0df861e6f3cb0da3c461066831d3165d449 Chris Anderson committed Sep 29, 2009
Showing with 861 additions and 0 deletions.
  1. +345 −0 LICENSE
  2. +89 −0 README
  3. +96 −0 bin/couchdb-ec2-env.sh
  4. +108 −0 bin/create-couchdb-image
  5. +50 −0 bin/launch-couchdb-instance
  6. +1 −0 bin/remote/cleanup-keys
  7. +9 −0 bin/remote/couchdb-init
  8. +133 −0 bin/remote/create-couchdb-image-remote
  9. +30 −0 bin/remote/motd
345 LICENSE

Large diffs are not rendered by default.

Oops, something went wrong.
89 README
@@ -0,0 +1,89 @@
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+
+Welcome to the couch.io CouchDB Amazon Machine Images. For more information please visit:
+
+ http://hosting.couch.io/ami/
+
+:: Launching an Image
+
+The hardest part of this is installing the EC2 command-line tools, and setting up the proper environment variables. The instructions for installing the EC2 tools are here:
+
+http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351&categoryID=88
+
+To set th environment variables, visit ./bin/couchdb-ec2-env.sh
+
+If you don't want deal with command line stuff, you can launch CouchDB instances from the couch.io web page:
+
+ http://hosting.couch.io/ami/
+
+The easiest part is launching an instance:
+
+ ./bin/launch-couchdb-instance
+
+This will prompt and error, if there are missing environment variables. Once it works, it will spit out the command you need to run to login to your instance.
+
+By default we'll launch a Pro AMI but you can configure the environment to use a public CouchDB AMI. The public AMIs are only slightly cheaper, so we hope you'll use a Pro AMI, so that some of your money goes to couch.io, instead of sending it all to Amazon.
+
+To purchase a pro AMI, visit one of these links:
+
+CouchDB 0.9.1
+ 32bit
+https://aws-portal.amazon.com/gp/aws/user/subscription/index.html?offeringCode=914F6020
+
+ 64bit
+https://aws-portal.amazon.com/gp/aws/user/subscription/index.html?offeringCode=BA504E0D
+
+CouchDB 0.10.0
+ 32bit
+https://aws-portal.amazon.com/gp/aws/user/subscription/index.html?offeringCode=FF8C4E12
+
+ 64bit
+https://aws-portal.amazon.com/gp/aws/user/subscription/index.html?offeringCode=253D99CD
+
+To switch to a public AMI, set PAID in the couchdb-ec2-env.sh to "public". Thanks for your support!
+
+:: Backups
+
+These AMIs are configured to store CouchDB data on the internal, ephemeral storage. This means when your instance terminates, all data will be lost. There are no backups. If you need backups, be sure to set them up. (We plan to add everything from auto-backups to clustering, in future iterations.)
+
+:: Security
+
+CouchDB by default is open for all (yay, admin party!), but that's probably not what you want, make sure to secure your CouchDB instance if you plan to use it in production. More information can be found in the Security Chapter of the CouchBook.
+
+To access CouchDB, you'll need to make sure that port 5984 is opened on the AWS firewall for your instance's security group. The documentation for opening ports is maintained by Amazon here. A smart way to control accees (this is how Chris's Blog works) is to set up a security group only available to trusted IP addresses, and then open port 5984 on that. If you want the whole world to be able to read your CouchDB, (which is fine for experimenting) the simple command is:
+
+ ec2-authorize -p 5984 default
+
+This will open up port 5984 for all instances you boot into the default group. Similarly (some EC2 101 here) if you want to ssh into your CouchDB box, run:
+
+ ec2-authorize -p 22 default
+
+Don't forget to close port 5984 when you are done:
+
+ ec2-revoke -p 5984 default
+
+
+:: Creating a Custom Image ::
+
+To create a new instance, edit the S3_BUCKET in couchdb-ec2-env.sh, as well as select the CouchDB version you want to install. Then run:
+
+ ./bin/create-couchdb-instance
+
+
+:: Thanks
+
+These scripts are based heavily on work by Eric Hammond, available in the Hadoop project.
+
+History:
+
+ 2008-05-16 Eric Hammond <ehammond@thinksome.com>
+ - Initial version including code from Kim Scheibel, Jorge Oliveira
+ 2008-08-06 Tom White
+ - Updated to use mktemp on fedora
+ 2009-09-29 Chris Anderson <jchris@couch.io>
+ - Updated for Ubuntu and CouchDB
+
+License: Apache 2.0
+ http://www.apache.org/licenses/LICENSE-2.0
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# utility functions
+require_var()
+{
+ var=$1
+ name=$2
+ if [ -z "$var" ]; then
+ echo "ERROR:"
+ echo " Missing environment variable \"$name\""
+ echo ""
+ echo "Exiting"
+ exit 2
+ fi
+}
+
+# COUCHDB_SVN=http://svn.apache.org/repos/asf/couchdb/tags/0.9.1
+# COUCHDB_VERSION=0.9.1
+
+COUCHDB_SVN=http://svn.apache.org/repos/asf/couchdb/branches/0.10.x
+COUCHDB_VERSION=0.10.x
+
+PAID="pro"
+# PAID="public"
+
+# The EC2 instance type: m1.small, m1.large, m1.xlarge
+INSTANCE_TYPE="m1.small"
+# INSTANCE_TYPE="m1.large"
+#INSTANCE_TYPE="m1.xlarge"
+
+# set this to your own bucket if you are custom building one
+S3_BUCKET=couchio-couchdb-$PAID
+
+require_var "$EC2_KEY_NAME" "EC2_KEY_NAME"
+require_var "$EC2_PRIVATE_KEY" "EC2_PRIVATE_KEY"
+require_var "$AWS_ACCOUNT_ID" "AWS_ACCOUNT_ID"
+require_var "$AWS_ACCESS_KEY_ID" "AWS_ACCESS_KEY_ID"
+require_var "$AWS_ACCESS_KEY_ID" "AWS_ACCESS_KEY_ID"
+
+EC2_KEYDIR=`dirname "$EC2_PRIVATE_KEY"`
+echo "using EC2_KEYDIR $EC2_KEYDIR"
+
+# Where your EC2 private key is stored (created when following the Amazon Getting Started guide).
+# You need to change this if you don't store this with your other EC2 keys.
+PRIVATE_KEY_PATH=`echo "$EC2_KEYDIR"/"id_rsa-$EC2_KEY_NAME"`
+echo "using ec2 ssh key: $PRIVATE_KEY_PATH"
+
+# TODO ensure id_rsa file permssions are 0600
+
+# SSH options used when connecting to EC2 instances.
+SSH_OPTS=`echo -i "$PRIVATE_KEY_PATH" -o StrictHostKeyChecking=no -o ServerAliveInterval=30`
+
+EC2_GROUP=default
+
+# SUPPORTED_ARCHITECTURES = ['i386', 'x86_64']
+if [ "$INSTANCE_TYPE" == "m1.small" ]; then
+ ARCH='i386'
+ BASE_AMI_IMAGE="ami-ed46a784" # http://alestic.com/ 32 bit Ubuntu 9.04
+else
+ ARCH='x86_64'
+ BASE_AMI_IMAGE="ami-5b46a732" # http://alestic.com/ 64 bit Ubuntu 9.04
+fi
+
+REQUIRED_BINARIES="\
+ ec2-describe-images \
+ ec2-describe-instances \
+ ec2-run-instances \
+ ec2-terminate-instances \
+ ec2-register \
+ ec2-deregister \
+ ec2-authorize"
+
+for binary in $REQUIRED_BINARIES; do
+ if [ ! -x "$EC2_HOME/bin/$binary" ]; then
+ echo "ERROR:"
+ echo " Command \"$binary\" not found. Did you install the EC2-Utilities?"
+ echo " Get them from:"
+ echo " http://docs.amazonwebservices.com/AWSEC2/latest/GettingStartedGuide/StartCLI.html"
+ echo ""
+ echo "Exiting."
+ exit 1
+ fi
+done
@@ -0,0 +1,108 @@
+#!/bin/sh
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Create a Hadoop AMI.
+# Inspired by Jonathan Siegel's EC2 script (http://blogsiegel.blogspot.com/2006/08/sandboxing-amazon-ec2.html)
+
+# Import variables
+bin=`dirname "$0"`
+bin=`cd "$bin"; pwd`
+. "$bin"/couchdb-ec2-env.sh
+
+AMI_IMAGE=`ec2-describe-images -a | grep $S3_BUCKET | grep $COUCHDB_VERSION | grep $ARCH | grep available | awk '{print $2}'`
+
+[ ! -z $AMI_IMAGE ] && echo "AMI already registered, use: ec2-deregister $AMI_IMAGE" && exit -1
+
+echo "will bundle to: $S3_BUCKET/couchdb-$COUCHDB_VERSION-$ARCH.manifest.xml"
+
+echo "Starting an AMI with ID $BASE_AMI_IMAGE."
+OUTPUT=`ec2-run-instances $BASE_AMI_IMAGE -k $EC2_KEY_NAME -t $INSTANCE_TYPE`
+BOOTING_INSTANCE=`echo $OUTPUT | awk '{print $6}'`
+
+echo "Instance is $BOOTING_INSTANCE."
+
+echo "Polling server status (ec2-describe-instances $BOOTING_INSTANCE)"
+while true; do
+ printf "."
+ HOSTNAME=`ec2-describe-instances $BOOTING_INSTANCE | grep running | awk '{print $4}'`
+ if [ ! -z $HOSTNAME ]; then
+ break;
+ fi
+ sleep 1
+done
+
+# todo check to see if port 22 is open on the security group
+# ec2-authorize -p 22 default
+
+echo "The server is available at $HOSTNAME. Will attempt to ssh in until it's fully booted."
+while true; do
+ REPLY=`ssh $SSH_OPTS "root@$HOSTNAME" 'echo "hello"'`
+ if [ ! -z $REPLY ]; then
+ break;
+ fi
+ sleep 5
+done
+
+TMP_ENV_FILE="$bin"/remote/tmp/ec2-env-tmp.sh
+echo "Creating temporary enviroment script in $TMP_ENV_FILE"
+mkdir -p "$bin"/remote/tmp
+echo "AWS_ACCOUNT_ID=$AWS_ACCOUNT_ID" > "$TMP_ENV_FILE"
+echo "AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID" >> "$TMP_ENV_FILE"
+echo "AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY" >> "$TMP_ENV_FILE"
+echo "AWS_ACCOUNT_ID=$AWS_ACCOUNT_ID" >> "$TMP_ENV_FILE"
+echo "COUCHDB_VERSION=$COUCHDB_VERSION" >> "$TMP_ENV_FILE"
+echo "COUCHDB_SVN=$COUCHDB_SVN" >> "$TMP_ENV_FILE"
+# echo "ERLANG_URL=$ERLANG_URL" >> "$TMP_ENV_FILE"
+echo "ARCH=$ARCH" >> "$TMP_ENV_FILE"
+echo "S3_BUCKET=$S3_BUCKET" >> "$TMP_ENV_FILE"
+echo "BUNDLE=$BUNDLE" >> "$TMP_ENV_FILE"
+
+echo "Copying scripts."
+
+# Copy setup scripts
+scp $SSH_OPTS "$TMP_ENV_FILE" "root@$HOSTNAME:/mnt"
+scp $SSH_OPTS "$bin"/remote/cleanup-keys "root@$HOSTNAME:/mnt"
+scp $SSH_OPTS "$bin"/remote/create-couchdb-image-remote "root@$HOSTNAME:/mnt"
+scp $SSH_OPTS "$bin"/remote/couchdb-init "root@$HOSTNAME:"
+scp $SSH_OPTS "$bin"/remote/motd "root@$HOSTNAME:/mnt"
+
+# Copy private key and certificate (for bundling image)
+scp $SSH_OPTS $EC2_KEYDIR/pk*.pem "root@$HOSTNAME:/mnt"
+scp $SSH_OPTS $EC2_KEYDIR/cert*.pem "root@$HOSTNAME:/mnt"
+
+echo "Deleting env script from local machine."
+rm "$TMP_ENV_FILE"
+
+# Connect to it, run the script
+ssh $SSH_OPTS "root@$HOSTNAME" '/mnt/create-couchdb-image-remote'
+
+# read -p "Login? [yes or No]: " answer
+#
+# if [ "$answer" == "yes" ]; then
+ # ssh $SSH_OPTS "root@$HOSTNAME"
+# fi
+
+ssh $SSH_OPTS "root@$HOSTNAME" '/mnt/cleanup-keys'
+
+# Register image
+echo "registering ami"
+ec2-register $S3_BUCKET/couchdb-$COUCHDB_VERSION-$ARCH.manifest.xml
+echo "registered: $S3_BUCKET/couchdb-$COUCHDB_VERSION-$ARCH.manifest.xml"
+
+
+echo "Terminating instance"
+ec2-terminate-instances $BOOTING_INSTANCE
@@ -0,0 +1,50 @@
+#!/bin/sh
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Launch an EC2 CouchDB instance.
+
+# Import variables
+bin=`dirname "$0"`
+bin=`cd "$bin"; pwd`
+. "$bin"/couchdb-ec2-env.sh
+
+# Get the name for the AMI Image
+AMI_IMAGE=`ec2-describe-images -a | grep $S3_BUCKET | grep $COUCHDB_VERSION | grep $ARCH | grep available | awk '{print $2}'`
+
+
+# Start a master
+echo "Starting instance with AMI $AMI_IMAGE"
+INSTANCE=`ec2-run-instances $AMI_IMAGE -n 1 -g $EC2_GROUP -k $EC2_KEY_NAME -t $INSTANCE_TYPE | grep INSTANCE | awk '{print $2}'`
+echo "Waiting for instance $INSTANCE to start"
+while true; do
+ printf "."
+ # get private dns
+ HOSTNAME=`ec2-describe-instances $INSTANCE | grep running | awk '{print $4}'`
+ if [ ! -z $HOSTNAME ]; then
+ echo "Started as $HOSTNAME"
+ break;
+ fi
+ sleep 1
+done
+
+# TODO move to launch
+echo "Access the Couch here:"
+echo "http://$HOSTNAME:5984/"
+echo "to login run:"
+echo "ssh $SSH_OPTS \"root@$HOSTNAME\""
+echo "to terminate run:"
+echo "ec2-terminate-instances $INSTANCE"
@@ -0,0 +1 @@
+rm -f /root/.ssh/authorized_keys
@@ -0,0 +1,9 @@
+#!/bin/sh
+
+# runs at boot
+# can be used to configure couchdb based on boot variables
+
+mkdir -p /mnt/couchdb/logs/
+mkdir -p /mnt/couchdb/data
+chown -R couchdb /mnt/couchdb
+
Oops, something went wrong.

0 comments on commit 0a97a0d

Please sign in to comment.