Merge 629c4c9 into 3d57af8

docs/best_practice/crate_ami.txt

@@ -0,0 +1,134 @@
+.. highlight:: yaml
+.. _crate_ami:
+
+================================
+Launch a custom Crate AMI on AWS
+================================
+
+Crate provides custom AMIs (see `AMI`_) that can be launched from every
+EC2 user by using the AWS Management Console or `AWS CLI`_. The naming
+convention is as follows:
+
+  ``Crate-AWS-AMI-<VERSION>-<REVISION>``
+
+Find Crate AMI
+==================
+
+To launch an own EC2 instance based on the Crate AMI use the search panel to
+to find the image on the public AMI repository. After choosing the desired AMI
+by its Crate version it can be launched as an EC2 instance running on your
+account.
+
+.. image:: ../_static/crate-ami-search.png
+   :alt: Search for public Crate AMI on AWS Management Console
+   :width: 100%
+
+
+If you are using the `AWS CLI`_ you can find the shared Crate AMI by using the
+``describe-images`` command. The easiest way to search for it is by its
+``AMI Name``. The following command lists every avaialble Crate AMI on the
+AWS platform.
+
+.. code-block:: bash
+
+    $ aws ec2 describe-images --filters "Name=name,Values=Crate-AWS-AMI-*"
+
+
+Launch the AMI
+==============
+
+A new EC2 instance can be started using the ``run-instances`` command. Use the
+available parameters of the command to configure its start-up behavior and
+hardware infrastructure.
+
+.. code-block:: bash
+
+    $ aws ec2 run-instances --image-id <AMI-ID> --count <NR-OF-INSTANCES> --instance-type <INSTANCE-TYPE> --user-data <USER-DATA>
+
+
+If you want to enable EC2 discovery on all instances to
+**support unicast host discovery** in order that every instance can join the
+cluster it is required to configure the AWS Credentials on startup of your
+instance. In this case the ``USER-DATA`` parameter has to be set with
+**Base64 encoded** data.
+
+.. code-block:: bash
+
+    #!/bin/bash
+    echo "
+    AWS_ACCESS_KEY_ID=###your_access_key_id###
+    AWS_SECRET_ACCESS_KEY='###your_secret_access_key###'
+    " >> /etc/sysconfig/crate
+
+
+.. note::
+    In case that you want to filter machines by instance tags or security groups
+    add this configuration also to the ``USER-DATA`` field parameter
+    (see :ref:`filter-by-tags`).
+
+
+Instance Types
+==============
+
+The instance type specifies the combination of CPU, memory, storage and
+networking capacity. To receive a better performance for running queries
+select an instance type which gives you the possibility to attach ephemeral
+storage. On newer instance types on AWS this storage is covered by
+`Solid-State-Drives`_ (short *SSD*). By choosing one of those instance types
+Crate will automatically mount and store its data on those devices to get a
+performance boost.
+Instance Types with additional instance store volumes (SSD or HDD) are
+currently all instances of type ``m3``, ``g2``, ``r3``, ``d2`` and ``i2``.
+
+To add a block device mapping before launching an instance it is possible to
+use the ``block-device-mappings`` parameter on the ``run-instances`` command.
+In this case ``ephemeral1`` will be added as an instance store volume with the
+device name ``/dev/sdc``.
+
+.. code-block:: bash
+
+    --block-device-mappings "[{\"DeviceName\": \"/dev/sdc\",\"VirtualName\":\"ephemeral1\"}]"
+
+.. note::
+    Note that the data stored in any kind of an instance storage type is not
+    permanent and it only persists in during the lifetime of an instance.
+
+.. _crate_ami_instance_type:
+
+
+Step 3: Configure Instance
+==========================
+
+This step allows you to set up the instance configuration in detail. It defines
+how many instances should be created and how the system behavoir should be
+during runtime.
+
+To support EC2 Discovery (see :ref:`ec2_setup`) in order that every instance can
+join the cluster the following settings must be set in the ``User Data``
+text-field in the ``Advanced Details`` section.
+
+This configuration script will be launched on startup of each instance. Use
+this settings and insert them into the `user data` text-field to provide
+EC2 discovery on the AWS network.
+
+.. code-block:: bash
+    :linenos:
+
+    #!/bin/bash
+    echo "
+    AWS_ACCESS_KEY_ID=###your_access_key_id###
+    AWS_SECRET_ACCESS_KEY='###your_secret_access_key###'
+    " >> /etc/sysconfig/crate
+
+    echo "
+    discovery.type: ec2
+    discovery.ec2:
+        groups: sg-crate-demo
+        tag.env: production
+        tag.type: database
+        availability_zones: us-west-1
+    " >> /etc/crate/crate.yml
+
+.. _AMI: https://en.wikipedia.org/wiki/Amazon_Machine_Image
+.. _Solid-State-Drives: https://en.wikipedia.org/wiki/Solid-state_drive
+.. _AWS CLI: https://aws.amazon.com/cli/

docs/best_practice/ec2_setup.txt

@@ -120,6 +120,8 @@ Optional Filters
 Sometimes however, you will want to have a more flexible setup. In this case
 there are a few other settings that can be set.
 
+.. _filter-by-tags:
+
 Filter by Tags
 --------------
 
@@ -175,5 +177,3 @@ See also :ref:`discovery.ec2.availability_zones <discovery_ec2_zones>`.
 .. _`Amazon EC2`: http://aws.amazon.com/ec2
 .. _`EC2 API`: http://docs.aws.amazon.com/AWSEC2/latest/APIReference/Welcome.html
 .. _`sign the requests`: http://docs.aws.amazon.com/general/latest/gr/signing_aws_api_requests.html
-
-

docs/best_practice/index.txt

@@ -16,3 +16,4 @@ This is a collection of best practice examples how to use Crate in the wild.
    migrating_from_mongodb
    multi_zone_setup
    ec2_setup
+   crate_ami