Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 105 lines (65 sloc) 3.38 kb
73519011 »
2010-05-13 initial commit
1 HBase EC2
2
3 This collection of scripts allows you to run HBase clusters on Amazon.com's Elastic Compute Cloud (EC2) service described at:
4
5 http://aws.amazon.com/ec2
6
7 To get help, type the following in a shell:
8
9 bin/hbase-ec2
10
11 You need both the EC2 API and AMI tools
12
13 http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351
14
15 http://developer.amazonwebservices.com/connect/entry.jspa?externalID=368&categoryID=88
16
17 installed and on the path.
18
19 When setting up keypairs on EC2, be sure to name your keypair as 'root'.
20
21 Quick Start:
22
23 1) Download and unzip the EC2 AMI and API tools zipfiles.
24
25 For Ubuntu, "apt-get install ec2-ami-tools ec2-api-tools".
26
27 2) Put the tools on the path and set EC2_HOME in the environment to point to
28 the top level directory of the API tools.
29
30 3) Configure src/contrib/ec2/bin/hbase-ec2-env.sh
31
32 Fill in AWS_ACCOUNT_ID with your EC2 account number.
33
34 Fill in AWS_ACCESS_KEY_ID with your EC2 access key.
35
36 Fill in AWS_SECRET_ACCESS_KEY with your EC2 secret access key.
37
38 Fill in EC2_PRIVATE_KEY with the location of your AWS private key file --
39 must begin with 'pk' and end with '.pem'.
40
41 Fill in EC2_CERT with the location of your AWS certificate -- must begin
42 with 'cert' and end with '.pem'.
43
44 Make sure the private part of your AWS SSH keypair exists in the same
45 directory as EC2_PRIVATE_KEY with the name id_rsa_root. Also, insure that
46 the permissions on the private key file are 600 (ONLY owner readable/
47 writable).
48
49 4) ./bin/hbase-ec2 launch-cluster <name> <nr-zoos> <nr-slaves>, e.g
50
51 ./bin/hbase-ec2 launch-cluster testcluster 3 3
52
53 5) Once the above command has finished without error, ./bin/hbase-ec2 login
54 <name>, e.g.
55
56 ./bin/hbase-ec2 login testcluster
57
58 6) Check that the cluster is up and functional:
59
60 hbase shell
61 > status 'simple'
62
63 You should see something like:
64
65 3 live servers
66 domU-12-31-39-09-75-11.compute-1.internal:60020 1258653694915
67 requests=0, regions=1, usedHeap=29, maxHeap=987
68 domU-12-31-39-01-AC-31.compute-1.internal:60020 1258653709041
69 requests=0, regions=1, usedHeap=29, maxHeap=987
70 domU-12-31-39-01-B0-91.compute-1.internal:60020 1258653706411
71 requests=0, regions=0, usedHeap=27, maxHeap=987
72 0 dead servers
73
74
75 Extra Packages:
76
77 It is possible to specify that extra packages be downloaded and installed on
78 demand when the master and slave instances start.
79
80 1. Set up a YUM repository. See: http://yum.baseurl.org/wiki/RepoCreate
81
82 2. Host the repository somewhere public. For example, build the
83 repository locally and then copy it up to an S3 bucket.
84
85 3. Create a YUM repository descriptor (.repo file). See:
86 http://yum.baseurl.org/wiki/RepoCreate
87
88 [myrepo]
89 name = MyRepo
90 baseurl = http://mybucket.s3.amazonaws.com/myrepo
91 enabled = 1
92
93 Upload the .repo file somewhere public, for example, in the root
94 directory of the repository,
95 mybucket.s3.amazonaws.com/myrepo/myrepo.repo
96
97 4. Configure hbase-ec2-env.sh thus:
98
99 EXTRA_PACKAGES="http://mybucket.s3.amazonaws.com/myrepo/myrepo.repo \
100 pkg1 pkg2 pkg3"
101
102 When the master and slave instances start, the .repo file will be added to
103 the Yum repository list and then Yum will be invoked to pull the packages
104 listed after the URL.
Something went wrong with that request. Please try again.