- Riak recipe updated to 1.0.2

  - Upgraded to Erlang R140B3
  - Removed riaksearch recipe as riak supports search in the same package now
  - Corrected download url
  - Attempted to optionize app.config so you can change backends, and other knobs easily with changing the attributes.

Signed-off-by: Scott M. Likens <slikens@engineyard.com>

cookbooks/main/recipes/default.rb

@@ -10,6 +10,9 @@
 #uncomment to turn on memcached
 # require_recipe "memcached"
 
+#uncomment ot run the riak recipe
+# require_recipe "riak"
+
 #uncomment to run the authorized_keys recipe
 #require_recipe "authorized_keys"
 

cookbooks/riak/README.md

@@ -1,4 +1,4 @@
-Riak Cookbook for EngineYard AppCloud
+Riak Cookbook for EngineYard EYCloud
 =========
 
 [Riak][1] is a Dynamo-inspired key/value store that scales predictably and easily. Riak also simplifies development by giving developers the ability to quickly prototype, test, and deploy their applications.
@@ -7,29 +7,29 @@ A truly fault-tolerant system, Riak has no single point of failure. No machines
 
 Don't forget to check out the [Riak Fast Track][9]!!!
 
-Special Note
---------
-
-You should *NOT* enable *BOTH* riak and riaksearch cookbooks at the same time.  Pick one solution or the other.
-
 Overview
 --------
 
-This cookbook once complete will attempt to provide one method of "Hosting" a Riak Ring on AppCloud.  It will not run inside your regular environment as this cookbook will attempt to achieve a scalable stable Riak configuration with the least disruption of automation possible.
+This cookbook once complete will attempt to provide one method of "Hosting" a Riak Ring on EYCloud.  It will not run inside your regular environment as this cookbook will attempt to achieve a scalable stable Riak configuration with the least disruption of automation possible.
 
 Design
 --------
 
 * 3+ utility instances 64-bit (m1.large +)
 
-* Riak 0.14 with Bitcask
-* Erlang R13B04 (installed from a custom binary package)
+* Riak 1.0.2 w/ defaults to leveldb
+* Erlang R140B3 (installed from a custom binary package)
 * haproxy is configured on 8097-8098 (pbc,http) with the http back-end using /ping to ensure the back-end is up.
 
+Protobuffer Notes
+--------
+
+Haproxy is configured in TCP Mode; if you use protobuffers you should either configure your clients directly to the server or configure a sane reconnect method as the connection will be stale after the connection timeout in haproxy.
+
 Notes
 --------
 
-This Cookbook automates the creation (join) action of a Riak 'Ring' on AppCloud.  As your needs may vary it is suggested to fork this recipe and make any customization you do on the fork.  You can omit the main cookbook it is only there for my testing purposes.
+This Cookbook automates the creation (join) action of a Riak 'Ring' on EYCloud.  As your needs may vary it is suggested to fork this recipe and make any customization you do on the fork.  You can omit the main cookbook it is only there for my testing purposes.
 
 Backups
 --------
@@ -46,7 +46,7 @@ This cookbook does not automate not facilitate any backup method currently.  The
 Benchmarks
 --------
 
-I [damm][4] have been benchmarking Riak on AppCloud for some time and have posted some of my tests with [basho_bench][5] for which you can review.  All posted results are using EBS as the diskstore, you can find better latency and speed by using the instance Ephemeral disks (/mnt) which can be [tuned][6] if you so wish.  *Note* you *MUST* use riak-admin to backup your data as it will *NOT* be stored on the EBS unit.  
+I [damm][4] have been benchmarking Riak on EYCloud for some time and have posted some of my tests with [basho_bench][5] for which you can review.  All posted results are using EBS as the diskstore, you can find better latency and speed by using the instance Ephemeral disks (/mnt) which can be [tuned][6] if you so wish.  *Note* you *MUST* use riak-admin to backup your data as it will *NOT* be stored on the EBS unit.  
 
 * You are free to enable the [basho_bench][7] [recipe][8] and then git clone git://github.com/basho/basho_bench.git to properly determine if your dataset / type would be a good fit for Riak.
 

cookbooks/riak/attributes/riak.rb

@@ -1,4 +1 @@
-riak :version => "0.14.2", :js_vm_count => "8", :js_max_vm_mem => "8"
-luwak :enabled => "true"
-bitcask :max_file_size => "16#80000000", :open_timeout => 4, :sync_strategy => "none", :merge_window => "always", :frag_merge_trigger => "60", :dead_bytes_merge_trigger => "536870912", :frag_threshold => "40", :dead_bytes_threshold => "134217728", :small_file_threshold => "10485760", :expiry_secs => "-1", :data_root => "data/bitcask"
-
+riak :version => "1.0.2", :data_root => "/data/riak/data", :map_js_vm_count => "8", :reduce_js_vm_count => "6", :hook_js_vm_count => "2", :js_max_vm_mem => "8", :js_thread_stack => "16", :riak_kv_stat => "true", :legacy_stats => "false", :legacy_keylisting => "false", :luwak_enabled => "false", :riaksearch_enabled => "false", :backend_enabled => "riak_kv_eleveldb_backend", :pb_backlog => "64"

cookbooks/riak/recipes/basho_bench.rb

@@ -3,8 +3,8 @@
 if File.exists?("/usr/bin/gfortran")
   # noop
 else
-  s3_forceinstall "sys-devel/gcc" do
-    version "4.1.2"
+  execute "emerge =sys-devel/gcc-4.1.2" do
+    action :run
   end
 end
 

cookbooks/riak/recipes/configure_riak.rb

@@ -1,6 +1,14 @@
 # Configure stuff goes here
 #
 
+directory "#{node[:riak][:data_root]}" do
+  action :create
+  owner "root"
+  group "root"
+  mode 0755
+  recursive true
+end
+
 template "/data/riak/etc/vm.args" do
   source "vm.args.erb"
   owner node[:owner_name]
@@ -16,19 +24,19 @@
   owner node[:owner_name]
   group node[:owner_name]
   variables(
-    :js_vm_count => node[:riak][:js_vm_count],
+    :riak_version => node[:riak][:version],
+    :map_js_vm_count => node[:riak][:map_js_vm_count],
+    :reduce_js_vm_count => node[:riak][:reduce_js_vm_count],
+    :hook_js_vm_count => node[:riak][:hook_js_vm_count],
     :js_max_vm_mem => node[:riak][:js_max_vm_mem],
-    :max_file_size => node[:bitcask][:max_file_size],
-    :open_timeout => node[:bitcask][:open_timeout],
-    :sync_strategy => node[:bitcask][:sync_strategy],
-    :merge_window => node[:bitcask][:merge_window], 
-    :frag_merge_trigger => node[:bitcask][:frag_merge_trigger],
-    :dead_bytes_merge_trigger => node[:bitcask][:dead_bytes_merge_trigger],
-    :frag_threshold => node[:bitcask][:frag_threshold],
-    :dead_bytes_threshold => node[:bitcask][:dead_bytes_threshold],
-    :small_file_threshold => node[:bitcask][:small_file_threshold],
-    :expiry_secs => node[:bitcask][:expiry_secs],
-    :data_root => node[:bitcask][:data_root],
-    :luwak => node[:luwak][:enabled]
+    :js_thread_stack => node[:riak][:js_thread_stack],
+    :riak_kv_stat => node[:riak][:riak_kv_stat],
+    :legacy_stats => node[:riak][:legacy_stats],
+    :legacy_keylisting => node[:riak][:legacy_keylisting],
+    :luwak_enabled => node[:riak][:luwak_enabled],
+    :riaksearch_enabled => node[:riak][:riaksearch_enabled],
+    :backend_enabled => node[:riak][:backend_enabled],
+    :pb_backlog => node[:riak][:pb_backlog],
+    :data_root => node[:riak][:data_root]
   )
 end

cookbooks/riak/recipes/default.rb

@@ -14,7 +14,7 @@
 
 # Install a version of Erlang (R130B4) provided by me.  I built the package, if you would like to see the ebuild sources make an issue.
   s3_install "dev-lang/erlang" do
-    version "13.2.4"
+    version "14.2.3"
   end
 end
 
@@ -30,7 +30,7 @@
 
 # Install a version of Erlang (R130B4) provided by me.  I built the package, if you would like to see the ebuild sources make an issue.
     s3_install "dev-lang/erlang" do
-      version "13.2.4"
+      version "14.2.3"
     end
 
 # Install python_setuptools as the version in portage is too old.

cookbooks/riak/recipes/install_riak.rb

@@ -1,8 +1,13 @@
 # Installer stuff here
 #
 
+package "dev-java/ant" do
+  version '1.7.0'
+  action :install
+end
+
 remote_file "/mnt/src/riak-#{node[:riak][:version]}.tar.gz" do
-  source "http://downloads.basho.com/riak/CURRENT/riak-#{node[:riak][:version]}.tar.gz"
+  source "http://downloads.basho.com.s3-website-us-east-1.amazonaws.com/riak/CURRENT/riak-#{node[:riak][:version]}.tar.gz"
   backup 0
   not_if { FileTest.exists?("/mnt/src/riak-#{node[:riak][:version]}.tar.gz") }
 end

cookbooks/riak/recipes/join_ring.rb

@@ -25,6 +25,8 @@
 
   execute "cd /data/riak;bin/riak-admin join riak@#{riak_hostname}" do
     action :run
+    epic_fail true
+    # FIXME: I need to start checking properly again, todo.
 #    not_if { ring_members.include?("riak@#{riak_hostname}") }
   end
 end