Permalink
Browse files

Merge branch 'warden'

* cf-release

Cliff Wang (16):
   Duplicate 'Add warden support for mongodb node' to new repo
   Merge 'services-r10'
   enforce loop-device style mongodb quota
   Fix warden rootfs install location
   Merge remote branch 'master' into 'warden'
   Use startup script to start mongodb server in warden
   Merge remote branch 'origin/master' into warden
   add switch disabling loop file quota
   Merge remote branch 'master' into 'warden'
   Merge "Merge remote branch 'master' into 'warden'" into warden
   Fix warden deployment and configure file to adapter warden server change
   Expand wardenlized services monit stop timeout
   Merge remote branch 'master' into 'warden'
   Wardenlized services deployment update
   Merge remote branch 'master' into 'warden'
   Merge "Expand wardenlized services monit stop timeout" into warden

Eagle Chen (1):
   Wardenize mysql

Frank Lu (1):
   rename jobs and packages of wardenzied services

Nicholas Kushmerick (5):
   Merge "Use startup script to start mongodb server in warden" into warden
   Merge " add vblob support for warden" into warden
   Merge "move vblob out of loop device" into warden
   point foo_gateway packages/jobs to ng
   Merge branch 'warden'

Tang Rui (12):
   Add warden support for redis node
   Merge "Add warden support for redis node" into warden
   Add warden support for rabbitmq node
   Use startup script to start redis server in warden
   Merge "Use startup script to start redis server in warden" into warden
   Add service_start_timeout option for mongodb, rabbitmq, redis and vblob service
   Merge "Add service_start_timeout option for mongodb, rabbitmq, redis and vblob service" into warden
   Create loop devices for disk quota
   Rename max_db_size to max_disk for wardenized services
   Merge branch 'master' into warden
   Update to latest services and vblob_src
   Merge branch 'master' into warden

Yang Yang (4):
   add vblob support for warden
   move vblob out of loop device
   prevent the racing condition where the chown of log directory was not finished
   abstract methods for wardenized services
   make vblob quota configurable

* src/warden

Kowshik Prakasam (3):
   Generate right help for 'run' command.
   Merge "Generate right help for 'run' command."
   Fix broken autocomplete in interactive repl v2.

Pieter Noordhuis (6):
   Use rspec options from .rspec file
   Generate default description from class name
   Change LimitBandwidth fields to be optional
   Bump warden-protocol
   Update warden-protocol, warden-client
   Merge "Clean container directories in /sys/fs/cgroup/<cpuacct, devices, memory>"

Tang Rui (1):
   Clean container directories in /sys/fs/cgroup/<cpuacct, devices, memory>

andyzh (1):
   implement bandwidth quota

* src/services

Andrew Liu (2):
   fix advertising message of service broker
   Merge "fix advertising message of service broker"

Cliff Wang (25):
   Merge remote branch 'origin/services-r9' into warden
   Move mongodb to warden
   enforce loop-device style mongodb quota
   Fix some errors introduced by auto merge
   Add integrity check in case mongod doesn't be shutdown properly; Add mongodb to warden migtration procedure;
   Merge "Add integrity check in case mongod doesn't be shutdown properly; Add mongodb to warden migtration procedure;" into warden
   MongoDB unit test refactor
   warden base gem
   Merge remote branch 'master' into 'warden'
   mongodb in warden serialization enabled
   warden base: 1. Add 'iptables' command retry 2. Replace class variables with class instance variables 3. Add log folder checking in case "bosh recreate" delete log folder 4. Enlong files copy timeout when doing migration to loop file per vblob 5. Refactor logger
   Merge "warden base: 1. Add 'iptables' command retry 2. Replace class variables with class instance variables 3. Add log folder checking in case "bosh recreate" delete log folder 4. Enlong files copy timeout when doing migration to loop file per vblob 5. Refactor logger" into warden
   Merge "mongodb in warden serialization enabled" into warden
   update base
   add switch disabling loop file quota
   Merge remote branch 'master' into 'warden'
   Update base ref
   Fix mongodb repair bug which making data file explode 4 times of its original size
   Fix repair path hardcode & a log message bug
   Merge commit 'master' into 'warden'
   Bump base reference to new warden protocol
   Instroduce multi-version to warden branch
   Using async warden stop interface to accelerate shutdown process
   Merge remote branch 'master' into 'warden'
   Bump warden refs

Eagle Chen (1):
   Wardenize mysql

Frank Lu (2):
   move wardenized services under ng
   Merge branch 'warden'

Harshawardhan Gadgil (4):
   Wardenize memcached
   Merge "Wardenize memcached" into warden
   Update base gem pointer to include provisioner check
   Refactor Marketplace Gateway and Appdirect code

Henry-CF (1):
   remove unused options arguments Signed-off-by: Henry-CF <hchen@rbcon.com>

Nicholas Kushmerick (4):
   Merge "Wardenize mysql" into warden
   Update base gem pointer for legacy services
   allow non-free mongo & memcache plans; fix memcache startup
   Merge changes I66390da8,I1350fb76

Tang Rui (17):
   Merge branch services-r11 into warden by squash
   Move redis to warden
   1. Grandfather existing instances that are too large; 2. Disable virtual memory settings; 3. Not mount migration directory since warden doesn't use it.
   1. Support snapshot and serialization API on warden; 2. Fix some bugs introduced by the merge from master and the new base.
   Move rabbitmq to warden
   Merge "Move rabbitmq to warden" into warden
   Add starting timeout configuration for unit test
   Fix two bugs for redis and rabbitmq unit tests
   Use base utility function to handle port and check instance start status
   Rename max_db_size to max_disk and some small enhancements
   Merge branch 'master' into warden
   Use multi-threads to stop instances
   Update base reference to fix duplicated run instance bug
   Return true when shutdown finishes
   Return true when rabbitmq node shutdown finishes
   Merge "Return true when rabbitmq node shutdown finishes" into warden
   Merge branch 'master' into warden

Yang Yang (11):
   move vblob into warden
   Merge "move vblob into warden" into warden
   remove vblob from loop device; add unit test cases on exceptions
   add include info to let vblob find proper error code
   fix some minor bugs for vblob
   let vblob raise proper exception on unprovision
   fix a bug in vblob migration unit test; avoid hard-codes in some unit-test
   Merge "fix a bug in vblob migration unit test; avoid hard-codes in some unit-test" into warden
   bump base pointer
   fork a process to remove vblob directory; fix some minor bugs in unit test
   Merge "fork a process to remove vblob directory; fix some minor bugs in unit test"

* src/tests

Nicholas Kushmerick (3):
   Don't assume 'free' plans
   fix blob limits tests
   don't assume service acls are set

Change-Id: Idb74619447e54c5a4b569d943e17f2d0b27bf304
  • Loading branch information...
2 parents e6ddbb4 + 478b0ba commit fbdb2aee29e56c361bbb901a33c234785da7bc17 @kushmerick kushmerick committed Oct 5, 2012
Showing with 3,904 additions and 31 deletions.
  1. +1 −1 jobs/memcached_gateway/templates/memcached_gateway_ctl
  2. +5 −0 jobs/memcached_node_ng/monit
  3. +18 −0 jobs/memcached_node_ng/spec
  4. +12 −0 jobs/memcached_node_ng/templates/logrotate.erb
  5. +2 −0 jobs/memcached_node_ng/templates/memcached_logrotate.cron.erb
  6. +47 −0 jobs/memcached_node_ng/templates/memcached_node.yml.erb
  7. +49 −0 jobs/memcached_node_ng/templates/memcached_node_ctl
  8. +25 −0 jobs/memcached_node_ng/templates/syslog_forwarder.conf.erb
  9. +1 −1 jobs/mongodb_gateway/templates/mongodb_gateway_ctl
  10. +33 −0 jobs/mongodb_node_ng/monit
  11. +32 −0 jobs/mongodb_node_ng/spec
  12. +12 −0 jobs/mongodb_node_ng/templates/logrotate.conf.erb
  13. +11 −0 jobs/mongodb_node_ng/templates/mongod_startup.sh
  14. +31 −0 jobs/mongodb_node_ng/templates/mongodb.conf.erb
  15. +18 −0 jobs/mongodb_node_ng/templates/mongodb_backup.cron.erb
  16. +45 −0 jobs/mongodb_node_ng/templates/mongodb_backup.erb
  17. +31 −0 jobs/mongodb_node_ng/templates/mongodb_backup.yml.erb
  18. +2 −0 jobs/mongodb_node_ng/templates/mongodb_logrotate.cron.erb
  19. +35 −0 jobs/mongodb_node_ng/templates/mongodb_migration_util.erb
  20. +60 −0 jobs/mongodb_node_ng/templates/mongodb_node.yml.erb
  21. +74 −0 jobs/mongodb_node_ng/templates/mongodb_node_ctl
  22. +50 −0 jobs/mongodb_node_ng/templates/mongodb_worker.yml.erb
  23. +39 −0 jobs/mongodb_node_ng/templates/mongodb_worker_ctl
  24. +8 −0 jobs/mongodb_node_ng/templates/services.conf
  25. +25 −0 jobs/mongodb_node_ng/templates/syslog_forwarder.conf.erb
  26. +19 −0 jobs/mongodb_node_ng/templates/warden.yml.erb
  27. +30 −0 jobs/mongodb_node_ng/templates/warden_ctl
  28. +1 −1 jobs/mysql_gateway/templates/mysql_gateway_ctl
  29. +49 −0 jobs/mysql_node_ng/monit
  30. +36 −0 jobs/mysql_node_ng/spec
  31. +139 −0 jobs/mysql_node_ng/templates/create_mysql_tmp_dir.erb
  32. +6 −0 jobs/mysql_node_ng/templates/my.bootstrap.erb
  33. +55 −0 jobs/mysql_node_ng/templates/my.cnf.erb
  34. +5 −0 jobs/mysql_node_ng/templates/my.shutdown.erb
  35. +18 −0 jobs/mysql_node_ng/templates/mysql_backup.cron.erb
  36. +45 −0 jobs/mysql_node_ng/templates/mysql_backup.erb
  37. +32 −0 jobs/mysql_node_ng/templates/mysql_backup.yml.erb
  38. +26 −0 jobs/mysql_node_ng/templates/mysql_ctl.erb
  39. +4 −0 jobs/mysql_node_ng/templates/mysql_init.erb
  40. +36 −0 jobs/mysql_node_ng/templates/mysql_migration_util.erb
  41. +64 −0 jobs/mysql_node_ng/templates/mysql_node.yml.erb
  42. +76 −0 jobs/mysql_node_ng/templates/mysql_node_ctl
  43. +14 −0 jobs/mysql_node_ng/templates/mysql_startup.sh
  44. +43 −0 jobs/mysql_node_ng/templates/mysql_worker.yml.erb
  45. +39 −0 jobs/mysql_node_ng/templates/mysql_worker_ctl
  46. +9 −0 jobs/mysql_node_ng/templates/services.conf
  47. +25 −0 jobs/mysql_node_ng/templates/syslog_forwarder.conf.erb
  48. +19 −0 jobs/mysql_node_ng/templates/warden.yml
  49. +39 −0 jobs/mysql_node_ng/templates/warden_ctl
  50. +6 −0 jobs/mysql_node_ng/templates/warden_mysql_init.erb
  51. +1 −1 jobs/postgresql_gateway/templates/postgresql_gateway_ctl
  52. +47 −0 jobs/postgresql_node_ng/monit
  53. +27 −0 jobs/postgresql_node_ng/spec
  54. +12 −0 jobs/postgresql_node_ng/templates/logrotate.conf.erb
  55. +18 −0 jobs/postgresql_node_ng/templates/postgresql_backup.cron.erb
  56. +45 −0 jobs/postgresql_node_ng/templates/postgresql_backup.erb
  57. +31 −0 jobs/postgresql_node_ng/templates/postgresql_backup.yml.erb
  58. +123 −0 jobs/postgresql_node_ng/templates/postgresql_ctl.erb
  59. +2 −0 jobs/postgresql_node_ng/templates/postgresql_logrotate.cron.erb
  60. +36 −0 jobs/postgresql_node_ng/templates/postgresql_migration_util.erb
  61. +68 −0 jobs/postgresql_node_ng/templates/postgresql_node.yml.erb
  62. +83 −0 jobs/postgresql_node_ng/templates/postgresql_node_ctl
  63. +47 −0 jobs/postgresql_node_ng/templates/postgresql_worker.yml.erb
  64. +35 −0 jobs/postgresql_node_ng/templates/postgresql_worker_ctl
  65. +23 −0 jobs/postgresql_node_ng/templates/services.conf
  66. +25 −0 jobs/postgresql_node_ng/templates/syslog_forwarder.conf.erb
  67. +19 −0 jobs/postgresql_node_ng/templates/warden.yml
  68. +47 −0 jobs/postgresql_node_ng/templates/warden_ctl
  69. +82 −0 jobs/postgresql_node_ng/templates/warden_postgresql_ctl
  70. +1 −1 jobs/rabbit_gateway/templates/rabbit_gateway_ctl
  71. 0 jobs/rabbit_node_ng/monit
  72. +25 −0 jobs/rabbit_node_ng/spec
  73. +12 −0 jobs/rabbit_node_ng/templates/logrotate.conf.erb
  74. +12 −0 jobs/rabbit_node_ng/templates/monit.erb
  75. +2 −0 jobs/rabbit_node_ng/templates/rabbit_logrotate.cron.erb
  76. +33 −0 jobs/rabbit_node_ng/templates/rabbit_migration_util.erb
  77. +49 −0 jobs/rabbit_node_ng/templates/rabbit_node.yml.erb
  78. +57 −0 jobs/rabbit_node_ng/templates/rabbit_node_ctl
  79. +23 −0 jobs/rabbit_node_ng/templates/rabbitmq_startup.sh
  80. +8 −0 jobs/rabbit_node_ng/templates/services.conf
  81. +25 −0 jobs/rabbit_node_ng/templates/syslog_forwarder.conf.erb
  82. +19 −0 jobs/rabbit_node_ng/templates/warden.yml.erb
  83. +34 −0 jobs/rabbit_node_ng/templates/warden_ctl
  84. +1 −1 jobs/redis_gateway/templates/redis_gateway_ctl
  85. +33 −0 jobs/redis_node_ng/monit
  86. +29 −0 jobs/redis_node_ng/spec
  87. +12 −0 jobs/redis_node_ng/templates/logrotate.conf.erb
  88. +18 −0 jobs/redis_node_ng/templates/redis_backup.cron.erb
  89. +45 −0 jobs/redis_node_ng/templates/redis_backup.erb
  90. +19 −0 jobs/redis_node_ng/templates/redis_backup.yml.erb
  91. +2 −0 jobs/redis_node_ng/templates/redis_logrotate.cron.erb
  92. +38 −0 jobs/redis_node_ng/templates/redis_migration_util.erb
  93. +49 −0 jobs/redis_node_ng/templates/redis_node.yml.erb
  94. +73 −0 jobs/redis_node_ng/templates/redis_node_ctl
  95. +8 −0 jobs/redis_node_ng/templates/redis_startup.sh
  96. +42 −0 jobs/redis_node_ng/templates/redis_worker.yml.erb
  97. +39 −0 jobs/redis_node_ng/templates/redis_worker_ctl
  98. +8 −0 jobs/redis_node_ng/templates/services.conf
  99. +25 −0 jobs/redis_node_ng/templates/syslog_forwarder.conf.erb
  100. +19 −0 jobs/redis_node_ng/templates/warden.yml.erb
  101. +29 −0 jobs/redis_node_ng/templates/warden_ctl
  102. +1 −1 jobs/vblob_gateway/templates/vblob_gateway_ctl
  103. +12 −0 jobs/vblob_node_ng/monit
  104. +23 −0 jobs/vblob_node_ng/spec
  105. +12 −0 jobs/vblob_node_ng/templates/logrotate.conf.erb
  106. +8 −0 jobs/vblob_node_ng/templates/services.conf
  107. +25 −0 jobs/vblob_node_ng/templates/syslog_forwarder.conf.erb
  108. +2 −0 jobs/vblob_node_ng/templates/vblob_logrotate.cron.erb
  109. +48 −0 jobs/vblob_node_ng/templates/vblob_node.yml.erb
  110. +73 −0 jobs/vblob_node_ng/templates/vblob_node_ctl
  111. +11 −0 jobs/vblob_node_ng/templates/vblob_startup.sh
  112. +19 −0 jobs/vblob_node_ng/templates/warden.yml.erb
  113. +34 −0 jobs/vblob_node_ng/templates/warden_ctl
  114. +1 −1 packages/memcached_gateway/packaging
  115. +1 −1 packages/memcached_gateway/pre_packaging
  116. +1 −1 packages/memcached_gateway/spec
  117. +11 −0 packages/memcached_node_ng/packaging
  118. +4 −0 packages/memcached_node_ng/pre_packaging
  119. +8 −0 packages/memcached_node_ng/spec
  120. +1 −1 packages/mongodb_gateway/packaging
  121. +1 −1 packages/mongodb_gateway/pre_packaging
  122. +1 −1 packages/mongodb_gateway/spec
  123. +19 −0 packages/mongodb_node_ng/packaging
  124. +11 −0 packages/mongodb_node_ng/pre_packaging
  125. +10 −0 packages/mongodb_node_ng/spec
  126. +1 −0 packages/mysql/packaging
  127. +1 −0 packages/mysql/spec
  128. +1 −1 packages/mysql_gateway/packaging
  129. +1 −1 packages/mysql_gateway/pre_packaging
  130. +1 −1 packages/mysql_gateway/spec
  131. +17 −0 packages/mysql_node_ng/packaging
  132. +10 −0 packages/mysql_node_ng/pre_packaging
  133. +11 −0 packages/mysql_node_ng/spec
  134. +1 −1 packages/postgresql_gateway/packaging
  135. +1 −1 packages/postgresql_gateway/pre_packaging
  136. +1 −1 packages/postgresql_gateway/spec
  137. +18 −0 packages/postgresql_node_ng/packaging
  138. +10 −0 packages/postgresql_node_ng/pre_packaging
  139. +11 −0 packages/postgresql_node_ng/spec
  140. +1 −1 packages/rabbit_gateway/packaging
  141. +1 −1 packages/rabbit_gateway/pre_packaging
  142. +1 −1 packages/rabbit_gateway/spec
  143. +17 −0 packages/rabbit_node_ng/packaging
  144. +11 −0 packages/rabbit_node_ng/pre_packaging
  145. +10 −0 packages/rabbit_node_ng/spec
  146. +1 −1 packages/redis_gateway/packaging
  147. +1 −1 packages/redis_gateway/pre_packaging
  148. +1 −1 packages/redis_gateway/spec
  149. +17 −0 packages/redis_node_ng/packaging
  150. +11 −0 packages/redis_node_ng/pre_packaging
  151. +10 −0 packages/redis_node_ng/spec
  152. +1 −1 packages/vblob_gateway/packaging
  153. +1 −1 packages/vblob_gateway/pre_packaging
  154. +1 −1 packages/vblob_gateway/spec
  155. +17 −0 packages/vblob_node_ng/packaging
  156. +11 −0 packages/vblob_node_ng/pre_packaging
  157. +10 −0 packages/vblob_node_ng/spec
  158. +49 −0 src/common/services/utils.sh
  159. +246 −0 src/mysql/mysql_warden.server
  160. +1 −1 src/services
  161. +1 −1 src/tests
  162. +1 −1 src/warden
@@ -24,7 +24,7 @@ case $1 in
/var/vcap/packages/syslog_aggregator/setup_syslog_forwarder.sh $JOB_DIR/config
<% end %>
- exec /var/vcap/packages/memcached_gateway/services/memcached/bin/memcached_gateway \
+ exec /var/vcap/packages/memcached_gateway/services/ng/memcached/bin/memcached_gateway \
-c $JOB_DIR/config/memcached_gateway.yml \
>>$LOG_DIR/memcached_gateway.stdout.log \
2>>$LOG_DIR/memcached_gateway.stderr.log
@@ -0,0 +1,5 @@
+check process memcached_node
+ with pidfile /var/vcap/sys/run/memcached_node/memcached_node.pid
+ start program "/var/vcap/jobs/memcached_node_ng/bin/memcached_node_ctl start"
+ stop program "/var/vcap/jobs/memcached_node_ng/bin/memcached_node_ctl stop"
+ group vcap
@@ -0,0 +1,18 @@
+---
+name: memcached_node_ng
+
+templates:
+ memcached_node_ctl: bin/memcached_node_ctl
+ memcached_node.yml.erb: config/memcached_node.yml
+
+ syslog_forwarder.conf.erb: config/syslog_forwarder.conf
+ logrotate.erb: config/logrotate.conf
+ memcached_logrotate.cron.erb: config/memcached_logrotate.cron
+
+packages:
+ - common
+ - memcached_node_ng
+ - memcached
+ - ruby
+ - sqlite
+ - syslog_aggregator
@@ -0,0 +1,12 @@
+<%
+ rotate = properties.memcached_node.logrotate && properties.memcached_node.logrotate.rotate
+ size = properties.memcached_node.logrotate && properties.memcached_node.logrotate.size
+%>
+/var/vcap/data/sys/service-log/*.log /var/vcap/data/sys/service-log/*/*.log /var/vcap/data/sys/service-log/*/*/*.log {
+ missingok
+ rotate <%= rotate || 3 %>
+ compress
+ delaycompress
+ copytruncate
+ size=<%= size || "50k" %>
+}
@@ -0,0 +1,2 @@
+<% freq_min=properties.memcached_node && properties.memcached_node.logrotate && properties.memcached_node.logrotate.freq_min || 5 %>
+*/<%= freq_min %> * * * * test -x /usr/sbin/logrotate && /usr/sbin/logrotate /var/vcap/jobs/memcached_node_ng/config/logrotate.conf >>/var/vcap/sys/log/memcached_node/memcached_logrotate_cron.log 2>&1
@@ -0,0 +1,47 @@
+---
+<%
+service = "memcached"
+plan_enabled = properties.service_plans && properties.service_plans.send(service.to_sym)
+plan = properties.plan || "free"
+plan_conf = plan_enabled && properties.service_plans.send(service.to_sym).send(plan.to_sym).configuration
+node = properties.memcached_node
+supported_versions = node.supported_versions
+default_version = node.default_version
+%>
+
+capacity: <%= plan_enabled && plan_conf.capacity || 16 %>
+plan: <%= plan %>
+local_db: sqlite3:/var/vcap/store/memcached/memcached_node.db
+mbus: nats://<%= properties.nats.user %>:<%= properties.nats.password %>@<%= properties.nats.address %>:<%= properties.nats.port %>
+index: <%= spec.index %>
+base_dir: /var/vcap/store/memcached/instances
+
+logging:
+ level: debug
+ file: /var/vcap/sys/log/memcached_node/memcached_node.log
+ <% if properties.syslog_aggregator %>
+ syslog: vcap.memcached_node
+ <% end %>
+
+pid: /var/vcap/sys/run/memcached_node/memcached_node.pid
+node_id: <%= "#{service}_node#{'_' + plan if plan != 'free'}_#{spec.index.to_i + 1}" %>
+supported_versions: <%= "['#{supported_versions.join("' , '")}']" %>
+default_version: <%= "'#{default_version}'"%>
+
+max_clients: <%= plan_enabled && plan_conf.max_clients || 500 %>
+memcached_server_path: /var/vcap/packages/memcached/bin/memcached
+port_range:
+ first: 11000
+ last: 25000
+
+z_interval: <%= properties.memcached_node.z_interval || 30 %>
+memcached_log_dir: /var/vcap/sys/service-log/memcached
+
+# TODO: max memory allocated to memcached service = capacity * memcached_memory = 16*64 = 1GB
+# Putting this to 1GB for testing, this *MUST* be appropriately changed
+memcached_memory: <%= properties.memcached_memory || 64 %>
+
+max_nats_payload: <%= properties.nats.max_payload || 1048576 %>
+memcached_start_timeout: <%= properties.memcached_node.memcached_start_timeout || 3 %>
+
+run_as_user: <%= properties.memcached_node.run_as_user || "" %>
@@ -0,0 +1,49 @@
+#!/bin/bash
+
+export PATH=/var/vcap/packages/ruby/bin:$PATH
+
+RUN_DIR=/var/vcap/sys/run/memcached_node
+LOG_DIR=/var/vcap/sys/log/memcached_node
+SERV_LOG_DIR=/var/vcap/sys/service-log/memcached
+PIDFILE=$RUN_DIR/memcached_node.pid
+JOB_DIR=/var/vcap/jobs/memcached_node_ng
+
+source /var/vcap/packages/common/utils.sh
+
+case $1 in
+
+ start)
+ pid_guard $PIDFILE "memcached node"
+
+ mkdir -p $RUN_DIR
+ mkdir -p $LOG_DIR
+ mkdir -p $SERV_LOG_DIR
+
+ echo $$ > $PIDFILE
+
+ <% if properties.syslog_aggregator %>
+ /var/vcap/packages/syslog_aggregator/setup_syslog_forwarder.sh $JOB_DIR/config
+ <% end %>
+
+ (crontab -l | sed /memcached_logrotate/d; cat /var/vcap/jobs/memcached_node_ng/config/memcached_logrotate.cron) | sed /^$/d | crontab
+
+ exec /var/vcap/packages/memcached_node_ng/services/ng/memcached/bin/memcached_node \
+ -c $JOB_DIR/config/memcached_node.yml \
+ >>$LOG_DIR/memcached_node.stdout.log \
+ 2>>$LOG_DIR/memcached_node.stderr.log
+
+ ;;
+
+ stop)
+ kill_and_wait $PIDFILE
+
+ crontab -l | sed /memcached_logrotate/d | crontab
+
+ ;;
+
+ *)
+ echo "Usage: memcached_node_ctl {start|stop}"
+
+ ;;
+
+esac
@@ -0,0 +1,25 @@
+<% if properties.syslog_aggregator %>
+$ModLoad imuxsock # local message reception (rsyslog uses a datagram socket)
+$MaxMessageSize 4k # default is 2k
+$WorkDirectory /var/vcap/sys/rsyslog/buffered # where messages should be buffered on disk
+
+# Forward vcap messages to the aggregator
+#
+$ActionResumeRetryCount -1 # Try until the server becomes available
+$ActionQueueType LinkedList # Allocate on-demand
+$ActionQueueFileName agg_backlog # Spill to disk if queue is full
+$ActionQueueMaxDiskSpace 32m # Max size for disk queue
+$ActionQueueLowWaterMark 2000 # Num messages. Assuming avg size of 512B, this is 1MiB.
+$ActionQueueHighWaterMark 8000 # Num messages. Assuming avg size of 512B, this is 4MiB. (If this is reached, messages will spill to disk until the low watermark is reached).
+$ActionQueueTimeoutEnqueue 0 # Discard messages if the queue + disk is full
+$ActionQueueSaveOnShutdown on # Save in-memory data to disk if rsyslog shuts down
+:programname, startswith, "vcap." @@<%= properties.syslog_aggregator.address %>:<%= properties.syslog_aggregator.port %>
+
+# Log vcap messages locally, too
+#$template VcapComponentLogFile, "/var/log/%programname:6:$%/%programname:6:$%.log"
+#$template VcapComponentLogFormat, "%timegenerated% %syslogseverity-text% -- %msg%\n"
+#:programname, startswith, "vcap." -?VcapComponentLogFile;VcapComponentLogFormat
+
+# Prevent them from reaching anywhere else
+:programname, startswith, "vcap." ~
+<% end %>
@@ -24,7 +24,7 @@ case $1 in
/var/vcap/packages/syslog_aggregator/setup_syslog_forwarder.sh $JOB_DIR/config
<% end %>
- exec /var/vcap/packages/mongodb_gateway/services/mongodb/bin/mongodb_gateway \
+ exec /var/vcap/packages/mongodb_gateway/services/ng/mongodb/bin/mongodb_gateway \
-c $JOB_DIR/config/mongodb_gateway.yml \
>>$LOG_DIR/mongodb_gateway.stdout.log \
2>>$LOG_DIR/mongodb_gateway.stderr.log
@@ -0,0 +1,33 @@
+check process warden
+ with pidfile /var/vcap/sys/run/warden/warden.pid
+ start program "/var/vcap/jobs/mongodb_node_ng/bin/warden_ctl start"
+ stop program "/var/vcap/jobs/mongodb_node_ng/bin/warden_ctl stop"
+ group vcap
+
+check process mongodb_node
+ with pidfile /var/vcap/sys/run/mongodb_node/mongodb_node.pid
+ start program "/var/vcap/jobs/mongodb_node_ng/bin/mongodb_node_ctl start"
+ stop program "/var/vcap/jobs/mongodb_node_ng/bin/mongodb_node_ctl stop"
+ group vcap
+ depends on warden
+
+<%
+service = "mongodb"
+plan_enabled = properties.service_plans && properties.service_plans.send(service.to_sym)
+plan = properties.plan || "free"
+plan_conf = plan_enabled && properties.service_plans.send(service.to_sym).send(plan.to_sym).configuration
+if properties.service_lifecycle and plan_enabled and plan_conf.lifecycle and plan_conf.lifecycle.enable
+ worker_count = plan_conf.lifecycle.worker_count || 1
+ worker_count.times do |ind|
+%>
+check process mongodb_worker_<%= ind %>
+ with pidfile /var/vcap/sys/run/mongodb_node/mongodb_worker_<%= ind %>.pid
+ start program "/var/vcap/jobs/mongodb_node_ng/bin/mongodb_worker_ctl start <%= ind %>"
+ stop program "/var/vcap/jobs/mongodb_node_ng/bin/mongodb_worker_ctl stop <%= ind %>"
+ group worker
+ group vcap
+ depends on mongodb_node
+<%
+ end
+end
+%>
@@ -0,0 +1,32 @@
+---
+name: mongodb_node_ng
+
+templates:
+ mongodb_node_ctl: bin/mongodb_node_ctl
+ mongodb_worker_ctl: bin/mongodb_worker_ctl
+ mongodb_migration_util.erb: bin/mongodb_migration_util
+
+ mongodb_node.yml.erb: config/mongodb_node.yml
+ mongodb_worker.yml.erb: config/mongodb_worker.yml
+ mongodb_backup.yml.erb: config/mongodb_backup.yml
+ mongodb_backup.cron.erb: config/mongodb_backup.cron
+ mongodb_backup.erb: config/mongodb_backup
+
+ syslog_forwarder.conf.erb: config/syslog_forwarder.conf
+ logrotate.conf.erb: config/logrotate.conf
+ mongodb_logrotate.cron.erb: config/mongodb_logrotate.cron
+
+ warden_ctl: bin/warden_ctl
+ warden.yml.erb: config/warden.yml
+ mongodb.conf.erb: config/mongodb.conf
+ mongod_startup.sh: bin/mongod_startup.sh
+ services.conf: config/services.conf
+
+packages:
+ - common
+ - mongodb_node_ng
+ - mongodb18
+ - mongodb20
+ - ruby
+ - sqlite
+ - syslog_aggregator
@@ -0,0 +1,12 @@
+<%
+ rotate = properties.mongodb_node.logrotate && properties.mongodb_node.logrotate.rotate
+ size = properties.mongodb_node.logrotate && properties.mongodb_node.logrotate.size
+%>
+/var/vcap/data/sys/service-log/*.log /var/vcap/data/sys/service-log/*/*.log /var/vcap/data/sys/service-log/*/*/*.log {
+ missingok
+ rotate <%= rotate || 3 %>
+ compress
+ delaycompress
+ copytruncate
+ size=<%= size || "50k" %>
+}
@@ -0,0 +1,11 @@
+#!/bin/bash
+for n in `seq 1 100`; do
+ if [ -f /tmp/vcap_chown.out ]; then
+ break
+ fi
+ sleep 0.1
+done
+
+touch /store/log/mongodb.log
+mkdir -p /store/instance/data
+exec /usr/bin/mongod --config /etc/mongodb.conf
@@ -0,0 +1,31 @@
+<%
+service = "mongodb"
+plan_enabled = properties.service_plans
+plan = properties.plan || "free"
+plan_conf = plan_enabled && properties.service_plans.send(service.to_sym).send(plan.to_sym).configuration
+%>
+dbpath = /store/instance/data
+logpath = /store/log/mongodb.log
+logappend = false # This doesn't works
+
+bind_ip = 0.0.0.0
+port = 27017
+
+auth = true
+
+nohttpinterface = true
+
+maxConns = <%= plan_enabled && plan_conf.max_clients || 500 %>
+
+# mms-token = 33f75ccb-ec85-4294-b6ec-e6ae7a63901f
+
+# journaling is currently preallocating 4G of space on disk
+# we can't do that.
+#
+#journal = true
+noprealloc = true
+
+# smallfiles = start first container file at 16MB, doubling 4 times.
+# so this = 240MB per user. (i.e. 16MB container + 32MB + 64MB + 128MB)
+#quota = true
+smallfiles = true
@@ -0,0 +1,18 @@
+<%
+service = "mongodb"
+cron_time = "0 5 * * *"
+interval = 5
+
+plan_enabled = properties.service_plans && properties.service_plans.send(service.to_sym)
+plan = properties.plan || "free"
+plan_conf = plan_enabled && properties.service_plans.send(service.to_sym).send(plan.to_sym).configuration
+backup_enabled = plan_enabled && plan_conf.backup && plan_conf.backup.enable
+if backup_enabled
+ cron_time = plan_conf.backup.interval || cron_time
+ tmp = plan_conf.backup.node_interval
+ interval = tmp && tmp.between?(1,60) ? tmp : interval
+end
+backup_cron = cron_time.sub(/^.+? /, "%d " % ((spec.index * interval) % 60 ))
+%>
+
+<%= backup_cron %> /var/vcap/packages/ruby/bin/ruby /var/vcap/packages/mongodb_node_ng/services/ng/mongodb/bin/mongodb_backup -c /var/vcap/jobs/mongodb_node_ng/config/mongodb_backup.yml >>/var/vcap/sys/log/mongodb_node/mongodb_backup_cron.log 2>&1
@@ -0,0 +1,45 @@
+#!/bin/bash
+<%
+service = "mongodb"
+plan_enabled = properties.service_plans && properties.service_plans.send(service.to_sym)
+plan = properties.plan || "free"
+plan_conf = plan_enabled && properties.service_plans.send(service.to_sym).send(plan.to_sym).configuration
+
+backup_enabled = plan_enabled && plan_conf.backup && plan_conf.backup.enable
+service_backup = properties.service_backup
+nfs_server = service_backup && service_backup.nfs_server
+first_node_only = backup_enabled && plan_conf.backup.first_node_only
+%>
+
+<% if backup_enabled && nfs_server && (!first_node_only || spec.index == 0) %>
+PIDFILE=/var/vcap/sys/run/mongodb_node/mongodb_backup.pid
+MOUNT_POINT=<%= service_backup.mount_point || "/var/vcap/services_backup" %>
+MT_OPTS="-t nfs"
+MT_EXPORT=<%= nfs_server.address %>:<%= nfs_server.export_dir %>
+
+source /var/vcap/packages/common/utils.sh
+
+case $1 in
+ start)
+ mkdir -p $MOUNT_POINT
+ check_mount "$MT_OPTS" "$MT_EXPORT" "$MOUNT_POINT"
+ (crontab -l | sed /mongodb_backup/d; cat /var/vcap/jobs/mongodb_node_ng/config/mongodb_backup.cron) | sed /^$/d | crontab
+
+ ;;
+
+ stop)
+ crontab -l | sed /mongodb_backup/d | crontab
+ kill_and_wait $PIDFILE
+ umount $MOUNT_POINT
+
+ ;;
+
+ *)
+ echo "Usage: mongodb_backup {start|stop}"
+
+ ;;
+
+esac
+<% else %>
+echo "Backup is disabled"
+<% end %>
Oops, something went wrong. Retry.

0 comments on commit fbdb2ae

Please sign in to comment.