Skip to content

Latest commit

 

History

History
128 lines (89 loc) · 6.05 KB

elasticsearch24.md

File metadata and controls

128 lines (89 loc) · 6.05 KB
Raw
menu title name description status
docs
parent
services
Elasticsearch 2.4
elasticsearch24
Elasticsearch version 2.4: a distributed, RESTful search and analytics engine
Deprecated

cloud.gov offers Elasticsearch 2.4 as a service.

Plans

Plan Name | Description | --------- | ----------- | ----- 1x | Elasticsearch instance with 1GB of RAM and 1 slice of CPU | 3x | Elasticsearch instance with 3GB of RAM and 1 slice of CPU | 6x | Elasticsearch instance with 6GB of RAM and 1 slice of CPU | 12x | Elasticsearch instance with 12GB of RAM and 2 slices of CPU | medium-ha | Elasticsearch cluster with three master nodes (3584M memory limit, 1792M heap) and three data nodes (3584M memory limit, 1792M heap, 10G disk) |

These instances are available in [sandbox spaces]({{< relref "overview/pricing/free-limited-sandbox.md#sandbox-limitations" >}}).

Note: The medium-ha plan runs multiple Elasticsearch master and data nodes on different machines, so that service is not interrupted by routine platform maintenance. The 1x, 3x, 6x, and 12x plans all run on a single instance and will be briefly unavailable during platform maintenance; these plans should not be used for production applications.

How to create an instance

To create a service instance run the following command:

cf create-service elasticsearch24 1x my-elastic-service

Additional notes

Elasticsearch 2.4 allows for dots in field names. This is a feature that existed pre-ElasticSearch 2.0 but was disabled in versions 2.0 to 2.3. Learn more about that here.

Shard/replica configuration for high availability

When using a high availability (HA) plan, indexes must be configured with a proper number of shards and replicas or it is still possible for your instance to experience downtime during platform maintenance.

All HA instances default to 3 shards and 2 replicas per index which is the recommended configuration.

Managing backups

Note: The Elasticsearch service does not currently have the ability to back up and restore your data. Data loss is possible in the event of a catastrophic failure at the infrastructure layer or user error (e.g., accidentally deleting your data).

The Elasticsearch service includes the AWS Cloud Plugin, which supports snapshot and restore with AWS S3. For detailed instructions, see the Snapshot and Restore Cloud AWS Repository documentation.

You can also use this example. The example assumes you already have an Elasticsearch service called my-elastic-service and an app called my-app, and that you have curl, jq, and the AWS command line interface available.

  • Create an instance of the [S3 service]({{< relref "docs/services/s3.md" >}}):

    cf create-service s3 basic my-s3-bucket

  • Create a service key to access S3 credentials:

    cf create-service-key my-s3-bucket my-key

  • Create a service key to access Elasticsearch credentials:

    cf create-service-key my-elastic-service my-key

  • Connect to your Elasticsearch service using port forwarding. Note: You'll need to leave the [cf ssh]({{< relref "docs/apps/using-ssh.md" >}}) command running and follow the next steps in a different terminal so that you can access the remote Elasticsearch instance from your local environment:

    es_credentials=$(cf service-key my-elastic-service my-key | tail -n +3)

es_hostname=$(echo "${es_credentials}" | jq -r '.hostname')
es_port=$(echo "${es_credentials}" | jq -r '.port')
es_username=$(echo "${es_credentials}" | jq -r '.username')
es_password=$(echo "${es_credentials}" | jq -r '.password')

cf ssh my-app -L "9200:${es_hostname}:${es_port}"

  • Create a snapshot repository:

    s3_credentials=$(cf service-key my-s3-bucket my-key | tail -n +3)

s3_bucket=$(echo "${s3_credentials}" | jq -r '.bucket')
s3_region=$(echo "${s3_credentials}" | jq -r '.region')
s3_access_key=$(echo "${s3_credentials}" | jq -r '.access_key_id')
s3_secret_key=$(echo "${s3_credentials}" | jq -r '.secret_access_key')

curl -X PUT -u "${es_username}:${es_password}" "localhost:9200/_snapshot/my_s3_repository" -d @<(cat <<EOF
{
  "type": "s3",
  "settings": {
    "bucket": "${s3_bucket}",
    "region": "${s3_region}",
    "access_key": "${s3_access_key}",
    "secret_key": "${s3_secret_key}"
  }
}
EOF
)

  • Create a snapshot:

    curl -X PUT -u "${es_username}:${es_password}" "localhost:9200/_snapshot/my_s3_repository/my_s3_snapshot"

  • Restore from a snapshot. Note: to restore to a new Elasticsearch instance, you'll need to connect to your new instance (see "Connect to your Elasticsearch service using port forwarding" above) and add the existing S3 repository (see "Create a snapshot repository" above).

    curl -X POST -u "${es_username}:${es_password}" "localhost:9200/_snapshot/my_s3_repository/my_s3_snapshot/_restore"

Rotating credentials

You can rotate credentials by creating a new instance and deleting the existing instance. If this is not an option, email cloud.gov support to request rotating the credentials manually.

The broker in GitHub

You can find the broker here: https://github.com/18F/kubernetes-broker.