Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

squashed merge of ccng-side-by-side to master

cloud_controller_ng shortlog c012225..c810a01

Jesse Zhang (50):
      MessageBus.subscribe with inbox
      Ignore yardoc rbenv and OS X metadata files
      Merge "MessageBus.subscribe with inbox"
      Legacy bulk api support
      Merge "Legacy bulk api support"
      custom json matcher
      Start and stop instances
      respond_with matcher
      Process Health Manager requests
      Minor fixes to doc / comments
      Validate state and package_state for an app
      update_uris for dea client
      Deserialize environment json on attribute read
      Mark for staging after touching app
      mark app for restage when BUNDLE_WITHOUT changes
      Refactor legacy app request translation
      Separate update and create request translations
      Fix typo in test case name
      Merge changes Idc3a26f0,Ie706a708,I36da969b,I0baf2c1a,I7ed32223,I423aebf2
      Merge changes I997e0c70,I7402710c,Ic89990aa
      Legacy apps env support
      Mandatory bulk_api in config
      Add test for cc_partition
      Merge "Legacy apps env support"
      Add test for legacy apps api read
      Decode UAA tokens when it looks like one
      [fix] Health Manager stop request
      Ensure app is staged before talking to DEA
      respond_with RSpec Matcher
      Merge changes I6092f7e2,Icf383aac
      [fix] Stub out app stager before each model test
      after_modify hook for model api
      dea_update_pending? on app objects
      Update DEA when an app's associated routes change
      Update DEA when a route's associated apps change
      nginx upload offloading
      nginx X-Accel for staged droplet download
      nginx offloading
      distinct end points for droplet download / upload
      Merge changes I95c91f7e,I30cde51d
      [tests] composable config-override
      Return 501 when gateways do not support lifecycle
      Use old files url when serving files from CC
      Properly encode tail param in query string
      Use local bind address
      World readable runtimes, frameworks and services
      serialization data service API support
      Require presence of auth token for gateways
      Authenticate requests to list_handles
      [fix] Fix list_handles authentication

Kowshik Prakasam (65):
      Implemented MessageBus#request.
      Added DeaClient.dea_request, .dea_publish methods.
      Implement DeaClient.find_specific_instance.
      Implement DeaClient.get_file_url
      Use '/' in exceptions when path is nil.
      Implement Files API.
      Add files wrapper for legacy apps.
      Add cc_partition flag to DEA start messages.
      Add timeout option to MessageBus.request.
      Implement DeaClient.find_instances.
      Minor styling fix.
      Define StatsError.
      Implement DeaClient.find_stats.
      Implement stats API.
      Styling fixes.
      Add stats wrapper for legacy apps.
      Improve test description.
      Clean up unit tests by using mock DeaClient.
      Check indices of instances in NATS response.
      Define class HealthManagerClient.
      DeaClient.find_instances allows MessageBus options
      Define InstancesError.
      Implement DeaClient.find_all_instances
      Implement V2 'instances' REST API endpoint.
      Add 'instances' wrapper for legacy apps.
      Fix styling.
      In files API, send path to DEA via NATS.
      Replace credentials with tuple.
      Fix styling.
      Add LegacyApiBase#has_default_space? method.
      Add helper to create user without default space.
      Don't return account usage if no default space.
      Fix typo.
      Add D100 filter for /info/services end-point.
      Prep files API for file server redirection.
      Use default values for credentials.
      Replace "url" with "uri".
      s/v1_api/redirect_ok.
      Remove obsolete comment.
      Fix bug that causes subscription on nil.
      Invert if block condition for better readability.
      In files API, redirect to file uri v2.
      Re-order tests to improve readability.
      Implement HealthManagerClient.find_crashes
      Implement crashes API.
      Add 'crashes' wrapper for legacy apps.
      Merge "Implement HealthManagerClient.find_crashes"
      Merge "Implement crashes API."
      Merge "Add 'crashes' wrapper for legacy apps."
      Refactor files API to improve readability.
      Catch HTTP 416 response from dea file server.
      Flag to override stopped state for app stats.
      Refactor v2 stats API to allow app stopped state.
      Allow stopped state in legacy stats API.
      Send email to service gateway during provision.
      Clear request guid in steno logging after request.
      Fix crashes API spec.
      Merge "Fix crashes API spec."
      Add play framework.
      Merge "Add play framework."
      Bump steno to include fix for ephemeral IO errors.
      NATS timeout in DeaClient.find_specific_instance
      Don't subscribe when no messages are expected.
      Publish droplet.updated when an app is updated.
      Make healthy_instances call work for multiple apps

Patrick Bozeman (220):
      remove service_id from service_instance
      accept empty attributes in an api message
      disallow space dev to update service binding
      provision/deprovision service instance with gw
      split label and version for legacy gw handle api
      bind and unbind on sg
      throw correct domain errors
      clarify spec desc for enumeration with perms
      refactored org permission checks
      remove duplicated method
      permissions for domains
      update new cc <-> gw message format
      require hm respondent
      pass mbus subscribe options down to nats
      v2 route api
      import/export routes in app
      Models::App uris method
      allow setting guid relations on unsaved objects
      legacy app api uri support
      send correct uris to dea on app start
      allow admins to create shared domains
      move migrator out of initial migration
      Model::Domain.default_serving_domain
      add default serving domain to org on create
      inherit shared domains from org on space creation
      switch legacy apps spec to use default domain
      add explicit org ownership to a route
      stronger relation validation for routes
      better translation of errors for InvalidRelations
      lookup route via visible routes in legacy apps
      fix env encoding for legacy app api
      send updated_at in hm bulk api
      add version to bulk hm api export
      add version check to hm bulk api spec
      correctly parse hm bulk api resume id
      use partition in hm subscription
      save app after updating droplet hash from stager
      correct encoding of id and token in bulk api
      fix app key_changed?
      send env as array with = to dea and stager
      Merge "send env as array with = to dea and stager"
      fix spec desc typo
      send quota event on service binding
      send quota calls on domains
      send quota events for routes
      hm client healthy_instances
      return running instances via legacy api
      fix message format of hm flapping status call
      yeti setup script
      :case_insensitive support for sequel Strings
      turn down debug logs for specs
      convert named things to be case insensitive
      Merge "turn down debug logs for specs"
      Merge "convert named things to be case insensitive"
      fix TEAM-59: add node08 to config
      fix TEAM-58: add legacy service offering endpoint
      Merge "fix TEAM-59: add node08 to config"
      fix TEAM-60: send 404 instead of 400
      fix team-61: allow model/stack during app create
      explict test for exception processing on the mbus
      pass staging client errors correctly back to ccng
      stop apps on deletion
      Merge "explict test for exception processing on the mbus"
      Merge "pass staging client errors correctly back to ccng"
      Merge "stop apps on deletion"
      Merge "[fix] Health Manager stop request"
      fix stopping instances vs indices in dea client
      send svc provider and vendor to dea on app start
      re-enable state on app create
      bump em to 1.0.0
      ruby 1.9.3 compatability changes
      AppStger.delete_droplet
      delete staged droplets on app deletion
      send old style svc label-version to stager
      include uaa token in security context
      legacy users api
      legacy get /services/:name
      support creation of N users for parallel yeti.
      add metadata to app model
      add command= to app model
      import command for apps and export metadata
      send app metadata to stager
      legacy app api stand alone command support
      remove metadata from v2 api
      add console support to app model
      add console support to dea client start msg
      add console support to app api
      range support for non-nginx version of files api
      Merge "range support for non-nginx version of files api"
      comment out dea v2 file url support
      get console support check correct
      Merge "comment out dea v2 file url support"
      Merge "get console support check correct"
      AppPackage.delete_package
      remove gratuitous stub in specs
      Merge "AppPackage.delete_package"
      Merge "remove gratuitous stub in specs"
      delete app packages on app deletion
      extra debug logging in hm respondent
      extra logging in resource pool
      don't leak exceptions into em defer
      Merge "nginx upload offloading"
      Merge "nginx X-Accel for staged droplet download"
      bump vcap_common
      Revert "nginx upload offloading"
      Revert "nginx X-Accel for staged droplet download"
      add basic varz for our sinatra apis
      Merge "s/v1_api/redirect_ok."
      Merge "Revert "nginx upload offloading""
      Merge "Revert "nginx X-Accel for staged droplet download""
      Merge "bump vcap_common"
      Merge "add basic varz for our sinatra apis"
      Merge "Fix bug that causes subscription on nil."
      basic varz support for sequel
      turn on varz for sequel
      fix db setup bug
      change sample config to default to localhost
      Merge "change sample config to default to localhost"
      don't abandon a qm token if we failed to commit it
      add vcap request id to thread local storage
      send vcap request id to qm during quota calls
      tell sequel gem to keep name, etc as strings
      send service config to gw when it enumerates handles
      Merge "tell sequel gem to keep name, etc as strings"
      Merge "send service config to gw when it enumerates handles"
      migrate svc instance and binding data to json
      fix/finalize legacy crashes api
      Merge "migrate svc instance and binding data to json"
      Merge "fix/finalize legacy crashes api"
      bump steno
      vcap ring buffer
      return ringbuffer or recent errors in varz
      extra debugging in base model controller
      optimize serialized relations_hash
      refactor spec env setup into a helper class
      speed up model specs
      move spec setting of quota mgr to always be blind
      remove reset_db from common api tests
      remove reset_db from permission tests
      remove reset_database from legacy_apps_spec
      remove reset_db from legacy_bulk before :each
      remove reset_db from legacy info
      speedup collection tests
      misc speedups for api unit tests
      dea client spec speedup
      minor timestamp spec speedup
      improved domain validation regex
      remove old template_obj from model specs
      make model deserialization spec helper more robust
      guard config for ccng
      add wildcard flag to domains
      calculate intermediate domains for a domain name
      overlapping domain validation checks
      keep relation depth in next/prev pagination urls
      use caller supplied request guid if it exists
      always paginate in a known order
      return error code 100 (in body) on bad auth token
      wildcard domain support for routes
      domain wildcard removal policy
      white space fixup
      fix domain permission check for system domains
      fix inverted wildcard logic
      refactor shared domain creation
      support multiple serving domains
      add shared_domains subset
      don't allow using the shared domains directly
      use destroy in specs to allow sequel hooks to run
      shared domain inheritance and visibility for orgs
      fix legacy route mapping
      Merge "Updated set_yeti to work with version UAA client 1.2.4"
      nil host route spec fixup
      move route ownership from org to space
      fix space domain test
      fix domain spec typo
      minor reorg of domain model spec
      add more domain auto-association tests
      change instance of spec delete to destroy
      better cleanup in collection specs and use shared domains in org spec
      auto-associate owned domains with their org
      Merge branch 'custom_domains'
      fix route migration for existing routes
      add billing flag to org
      optionaly disable model tests
      only run uniqueness tests if needed
      be explict with model requires
      initial billing event model
      make object serialization configurable
      initial billing events api
      remove old quota manager calls
      alias org billing_enabled?
      AppStartEvent.create_from_app
      AppStopEvent.create_from_app
      OrganizationStartEvent.create_from_org
      ServiceCreateEvent.create_from_service_instance
      ServiceDeleteEvent.create_from_service_instance
      emit app start/stop billing events
      emit service create/delete events
      emit org billing start events
      add required start/end date params to billing api
      s/id/guid for billing events
      org summary endpoint
      space summary endpoint
      app summary endpoint
      remove bad export on service plan
      Health and framework/runtime info for space summary
      add health to app summary
      fix return from healthy_instances
      enhance space space summary
      add new style runtime config file support
      add new style framework config file support
      switch to loading new format rt and fw
      remove old legacy frameworks config
      refactor DeaClient.get_file_uri
      don't expose v1 dea uri on a fetch failure
      extract get_file_uri_for_search_param in files api
      s/instance_id/instance in files_spec
      DeaClient.get_files_for_instance_id
      add instance_id search param to files api
      send fw and rt info in staging messages
      send fw info in dea messages

Ryan Tang (1):
      Updated set_yeti to work with version UAA client 1.2.4

dea_next shortlog ed1e551..1362250

Anfernee Gui (2):
      Generate private id for running instance
      Merge "Generate private id for running instance"

Kowshik Prakasam (33):
      Move deaClient and common code to separate files.
      Remove unwanted symbol exports.
      Ignore write errors when client has disconnected.
      Move file streaming API to separate file.
      Do case sensitive check on tail query parameter.
      Add comments.
      Re-order method definition for better readability.
      Merge "Move deaClient and common code to separate files."
      Merge "Remove unwanted symbol exports."
      Merge "Ignore write errors when client has disconnected."
      Merge "Move file streaming API to separate file."
      Merge "Do case sensitive check on tail query parameter."
      Merge "Add comments."
      Merge "Re-order method definition for better readability."
      Dont use pointers for slices.
      Change application_id from Integer to String.
      Deprecate application_users but preserve it in env
      Default value for path when generating file uri v2.
      Pass empty tags in dir server registration.
      Revert "Lazily fill runtimes hash"
      Use line comments instead of block comments.
      Fix comment.
      Access type http.Header through its API.
      Refactor to prepare code for byte range queries.
      Close file handles.
      Implement handler for http range queries.
      Better err message when runner cant read config.
      Improve comments.
      Merge branch 'master' into ccng-integration
      Fix seek constant.
      Merge branch 'master' into revert_runtime_changes
      Revert "Runtime messages now use the version_output attribute"
      Merge branch 'master' into revert_runtime_changes

Patrick Bozeman (1):
      Merge branch 'master' into ccng-integration

Pieter Noordhuis (38):
      Update vcap_common
      Update steno
      Remove unused code
      Use real and absolute path to tmpdir
      Add cc_partition attribute to instance
      Explicitly stringify environment pairs
      Update service schema
      Handle nils in runtime environments
      Runtime messages now use the version_output attribute
      Stringify droplet attribute for incoming messages
      Merge "Runtime messages now use the version_output attribute"
      Merge "Stringify droplet attribute for incoming messages"
      Stringify droplet attribute in more places
      Undo 75937ab0, fall back on 'instance_ids'
      Merge branch 'master' into ccng-integration
      Fix crash reaping
      Remove unused dependency
      Change crash reaper specs to hit the disk
      Schedule callback after reaping crash from thread
      Reap crashes when disk gets full
      Merge branch 'master' into ccng-integration
      Revert "Merge branch 'master' into ccng-integration"
      Update em-warden-client
      Close warden connections after destroy
      Load snapshot before connecting to NATS
      Compatibility with WIP version of warden
      Update eventmachine to 1.0.0 and use epoll
      Write spec logs to stderr
      Add coding directive
      Update em-http-request to 1.0.3
      Behavior of EM::Connection#error? changed
      Don't attempt to connect when health check is done
      Include reason in log message when crash is reaped
      Create crashes path at start time
      Update warden dependencies
      Update steno dependency
      Pass resource limits directly to warden
      Move generic warden functionality to base class

warden shortlog d8b0a93d6..b5ab5fc24

Bob Nugmanov (2):
      Whitelist safe devices
      Merge "Whitelist safe devices"

Kowshik Prakasam (9):
      Create work path before specs start.
      Resource limits for run and spawn commands.
      Plumb resource limits from warden server to wsh.
      Specify optional handle when creating a container.
      Introduce container id.
      Custom handle for create command.
      Return list of alive job ids in info response.
      Populate info response with list of alive job ids.
      Don't special case iomux-link exit status.

Pieter Noordhuis (77):
      First stab at warden shell
      Extract client_loop from client_accept
      Use CLOEXEC on pipes
      Dynamically allocate buffer space
      Enhance pump code
      Move hooks
      Change to right directory
      Remove unnecessary interpolation syntax
      Assert number of fds to receive
      Use extra fd for exit status
      Configure network interfaces from hooks
      Force link creation
      Update start/stop to use warden shell
      Mount /proc from hook
      Remove SSH related setup code
      Remove overrides now that upstart is no longer used
      Trim down setup.sh
      Move common.sh to lib/
      Change mount paths; include config from common.sh
      Ignore wshc and wshd
      Select for reads, block for writes
      Forward return value
      Skip fd extraction on EOF
      Skip fd extraction when pointer is not specified
      Add copy in case splice cannot be used
      Support interactive terminals
      Define buffer size in msg.h
      Fix number of fds to send to client
      Specify environment on exec
      Extract fork to separate function
      Forward argv to wshd
      Forward resource limits to wshd
      Remove unused files
      Rename wshc -> wsh
      Configurable process title for wshd
      Add run directory to skeleton
      Remove no longer required code from setup.sh
      Add option parsing to wsh to discard rsh options
      Update destroy script
      Fix fcntl error (F_SETFD vs F_SETFL)
      Set FD_CLOEXEC on client connection
      Undefine macro that is only used once
      Loop waitpid in wshd
      Forward user from RSH compatibility mode to wshd
      Add route for default gateway from hook
      Make stop script graceful again
      Fix regexp for matching tc output
      Modify server code to use wsh
      Remove the RLIMIT_RTTIME resource limit
      Copy wshd and wsh from setup task
      Break from waitpid loop when it -1 or 0
      Use -l flag to su to chdir to HOME
      Loop to write entire buffer
      Merge branch 'master' into wsh2
      Use /tmp/warden/spec as working directory for specs
      Use small buffer on stack instead of heap
      Change temporary directory used in container code
      Change user via wsh/wshd
      Fix setup task
      Use perror
      Setup rudimentary environment from wshd
      Pass logger to deferred child to preserve handle
      Use external utility to close non-default fds
      Update em-posix-spawn
      Merge branch 'master' into wsh2
      Update dependencies
      Combine resource limits in configuration and requests
      Remove unused code
      Close in/out/err in wshd when entering accept loop
      Merge remote-tracking branch 'origin/wsh2' into wsh2
      Merge branch 'wsh2'
      RLIMIT_AS must be specified in bytes
      Configure write side of pump pair to be blocking
      Merge "Configure write side of pump pair to be blocking"
      Remove dependency on vcap-common
      Update eventmachine to 1.0.0
      Whitelist ptty related devices

Yang Yang (1):
      make setquota optional

Change-Id: I2ebb9fae3fa7b16e1698a84b2edde5436c093350
  • Loading branch information...
commit e02f3386e2c694b57feb9da54d210b0713a6e95d 1 parent 5552d2c
@pbozeman pbozeman authored
Showing with 725 additions and 281 deletions.
  1. +4 −0 config/blobs.yml
  2. +48 −41 jobs/cloud_controller/templates/cloud_controller.yml.erb
  3. +6 −0 jobs/cloud_controller_ng/monit
  4. +18 −8 jobs/cloud_controller_ng/spec
  5. +35 −202 jobs/cloud_controller_ng/templates/cloud_controller_ng.yml.erb
  6. +16 −7 jobs/cloud_controller_ng/templates/cloud_controller_ng_ctl.erb
  7. +10 −0 jobs/cloud_controller_ng/templates/grails_staging.yml
  8. +10 −0 jobs/cloud_controller_ng/templates/idmapd.conf.erb
  9. +10 −0 jobs/cloud_controller_ng/templates/java_web_staging.yml
  10. +10 −0 jobs/cloud_controller_ng/templates/lift_staging.yml
  11. +73 −0 jobs/cloud_controller_ng/templates/mime.types
  12. +2 −0  jobs/cloud_controller_ng/templates/nfs-common
  13. +129 −0 jobs/cloud_controller_ng/templates/nginx.conf.erb
  14. +47 −0 jobs/cloud_controller_ng/templates/nginx_ctl
  15. +11 −0 jobs/cloud_controller_ng/templates/node_staging.yml
  16. +10 −0 jobs/cloud_controller_ng/templates/play_staging.yml
  17. +12 −0 jobs/cloud_controller_ng/templates/rack_staging.yml
  18. +11 −0 jobs/cloud_controller_ng/templates/rails3_staging.yml
  19. +150 −0 jobs/cloud_controller_ng/templates/runtimes.yml
  20. +12 −0 jobs/cloud_controller_ng/templates/sinatra_staging.yml
  21. +10 −0 jobs/cloud_controller_ng/templates/spring_staging.yml
  22. +18 −0 jobs/cloud_controller_ng/templates/standalone_staging.yml
  23. +1 −1  jobs/dea_next/spec
  24. +2 −2 jobs/dea_next/templates/dea_ctl
  25. +3 −3 jobs/dea_next/templates/warden_ctl
  26. +15 −8 jobs/health_manager/templates/health_manager.yml.erb
  27. +9 −3 jobs/health_manager_next/templates/health_manager_next.yml.erb
  28. +5 −3 jobs/nats/templates/nats.yml.erb
  29. +9 −0 jobs/postgres/templates/postgres_ctl.erb
  30. +20 −0 packages/ruby_next/packaging
  31. +6 −0 packages/ruby_next/spec
  32. +1 −1  src/cloud_controller_ng
  33. +1 −1  src/dea_next
  34. +1 −1  src/warden
View
4 config/blobs.yml
@@ -619,6 +619,10 @@ uaa/cloudfoundry-identity-uaa-1.2.5.war:
object_id: eyJvaWQiOiI0ZTRlNzhiY2ExMWUxMjIwMDRlNGU4ZWM2NDg0MzEwNTA4NzAx%0AMzgyMDVkMiIsInNpZyI6IjVmSEVRaFUvWjZLamRZNUo2TEVjcCt0d1Fndz0i%0AfQ==%0A
sha: f357c80c2febbfc5505e4d7c2e1edda0d29c3bf1
size: 17960086
+ruby/ruby-1.9.3-p194.tar.gz:
+ object_id: eyJvaWQiOiI0ZTRlNzhiY2E1MWUxMjIyMDRlNGU5ODYzZjI4ZjMwNTA1N2M0%0AZDgyYjRkMyIsInNpZyI6Ii9rY212b3JDYmFtNEJ2S1pJaEV5bGk2Q2lyST0i%0AfQ==%0A
+ sha: 31cf6bd981e4c929e5dc3bbdb341833eab1bd9f2
+ size: 12432239
dashboard/dashboard-1.0-SNAPSHOT-2b7dcaa1.war:
object_id: eyJvaWQiOiI0ZTRlNzhiY2EyMWUxMjIwMDRlNGU4ZWM2NDdhNTQwNTA4YWNk%0ANDJlMjI4ZSIsInNpZyI6Im9FcTdQaTBsb21HclJPSHJYQTh1YVI3L1RaZz0i%0AfQ==%0A
sha: 08004df2a5b2a7a7e610da4ac1eaaa5a0c14ad25
View
89 jobs/cloud_controller/templates/cloud_controller.yml.erb
@@ -1,34 +1,39 @@
---
-<% db = properties.ccdb.databases.find { |db| db.tag == "cc" } %>
-<% db_role = properties.ccdb.roles.find { |role| role.tag == "admin" } %>
+<% cc_props_name = properties.cc_props || 'cc' %>
+<% cc_props = properties.send(cc_props_name) %>
-external_uri: api.<%= properties.domain %>
+<% ccdb_prop_name = properties.db || 'ccdb' %>
+<% ccdb = properties.send(ccdb_prop_name) %>
+<% db_name = ccdb.databases.find { |db| db.tag == "cc" }.name %>
+<% db_role = ccdb.roles.find { |role| role.tag == "admin" } %>
+
+external_uri: <%= cc_props.external_uri || "api." + properties.domain %>
description: <%= properties.description || "VMware's Cloud Application Platform" %>
support_address: <%= properties.support_address || "http://support.cloudfoundry.com" %>
local_route: <%= spec.networks.send(properties.networks.apps).ip %>
-<% if properties.cc.allow_registration || properties.cc.allow_registration == nil %>
+<% if cc_props.allow_registration || cc_props.allow_registration == nil %>
allow_registration: true
<% else %>
allow_registration: false
<% end %>
external_port: 9022 #public CC port
-<% if properties.cc.max_droplet_size %>
-max_droplet_size: <%= properties.cc.max_droplet_size %>
+<% if cc_props.max_droplet_size %>
+max_droplet_size: <%= cc_props.max_droplet_size %>
<% end %>
bulk_api:
auth:
user: bulk_api
- password: <%= properties.cc.password %>
+ password: <%= cc_props.password %>
nginx:
- use_nginx: <%= properties.cc.use_nginx %> #nginx proxy in front of CC?
+ use_nginx: <%= cc_props.use_nginx %> #nginx proxy in front of CC?
instance_socket: /var/vcap/sys/run/cloud_controller/cloud_controller.sock
#insecure_instance_port: 9025 #NOT SECURE -- DEBUG ONLY!!!!
index: <%= spec.index %>
-index_page: <%= properties.cc.index_page %>
+index_page: <%= cc_props.index_page %>
info_file: /var/vcap/packages/cloud_controller/config/cloud_controller-info.json
# Legacy (FIXME REMOVE)
@@ -51,18 +56,18 @@ logging:
<% if properties.syslog_aggregator %>
syslog: vcap.cloud_controller
<% end %>
- level: <%= properties.cc.logging_level || "info" %>
+ level: <%= cc_props.logging_level || "info" %>
rails_logging:
file: /var/vcap/sys/log/cloud_controller/rails.log
- level: <%= properties.cc.rails_logging_level || "info" %>
+ level: <%= cc_props.rails_logging_level || "info" %>
event_logging:
file: /var/vcap/sys/log/cloud_controller/events.log
keys:
- password: <%= properties.cc.password %>
- token: <%= properties.cc.token %>
+ password: <%= cc_props.password %>
+ token: <%= cc_props.token %>
pid: /var/vcap/sys/run/cloud_controller/cloud_controller.pid
@@ -70,33 +75,35 @@ mbus: nats://<%= properties.nats.user %>:<%= properties.nats.password %>@<%= pro
rails_environment: production
-<% if properties.cc.new_initial_placement && spec.index.to_i == 0 %>
+<% if cc_props.new_initial_placement && spec.index.to_i == 0 %>
new_initial_placement: true
<% else %>
new_initial_placement: false
<% end %>
+cc_partition: <%= cc_props.cc_partition || "default" %>
+
database_environment:
production:
- database: <%= db.name %>
- host: <%= properties.ccdb.address %>
- port: <%= properties.ccdb.port %>
+ database: <%= db_name %>
+ host: <%= ccdb.address %>
+ port: <%= ccdb.port %>
encoding: utf8
username: <%= db_role.name %>
password: <%= db_role.password %>
- pool: <%= properties.ccdb.pool_size %>
+ pool: <%= ccdb.pool_size %>
timeout: 2000
template: template0 # Required for utf8 encoding
adapter: postgresql
-<% if properties.cc.uaa && properties.uaa.cc %>
+<% if cc_props.uaa && properties.uaa.cc %>
uaa:
- enabled: <%= properties.cc.uaa.enabled %>
+ enabled: <%= cc_props.uaa.enabled %>
url: https://uaa.<%= properties.domain %>
- resource_id: <%= properties.cc.uaa.resource_id %>
+ resource_id: <%= cc_props.uaa.resource_id %>
token_secret: <%= properties.uaa.cc.token_secret %>
client_secret: <%= properties.uaa.cc.client_secret %>
- token_creation_email_filter: <%= properties.cc.uaa.token_creation_email_filter %>
+ token_creation_email_filter: <%= cc_props.uaa.token_creation_email_filter %>
<% else %>
uaa:
enabled: false
@@ -116,38 +123,38 @@ redis:
<% end %>
staging:
- max_staging_runtime: <%= properties.cc.max_staging_runtime || 120 %>
+ max_staging_runtime: <%= cc_props.max_staging_runtime || 120 %>
auth:
- user: <%= properties.cc.staging_upload_user %>
- password: <%= properties.cc.staging_upload_password %>
+ user: <%= cc_props.staging_upload_user %>
+ password: <%= cc_props.staging_upload_password %>
-allow_debug: <%= properties.cc.allow_debug || false %>
+allow_debug: <%= cc_props.allow_debug || false %>
-admins: [ "<%= properties.cc.admins.join('", "') %>" ]
+admins: [ "<%= cc_props.admins.join('", "') %>" ]
-https_required: <%= properties.cc.https_required || false %>
-https_required_for_admins: <%= properties.cc.https_required_for_admins || false %>
+https_required: <%= cc_props.https_required || false %>
+https_required_for_admins: <%= cc_props.https_required_for_admins || false %>
-<% if properties.cc.admin_account_capacity %>
+<% if cc_props.admin_account_capacity %>
admin_account_capacity:
- memory: <%= properties.cc.admin_account_capacity.memory || 32 * 1024 %>
- app_uris: <%= properties.cc.admin_account_capacity.app_uris || 16 %>
- services: <%= properties.cc.admin_account_capacity.services || 32 %>
- apps: <%= properties.cc.admin_account_capacity.apps || 200 %>
+ memory: <%= cc_props.admin_account_capacity.memory || 32 * 1024 %>
+ app_uris: <%= cc_props.admin_account_capacity.app_uris || 16 %>
+ services: <%= cc_props.admin_account_capacity.services || 32 %>
+ apps: <%= cc_props.admin_account_capacity.apps || 200 %>
<% end %>
# Normal users limited to 512M, 4 Services, and 4 URIs per App
default_account_capacity:
-<% unless properties.cc.default_account_capacity %>
+<% unless cc_props.default_account_capacity %>
memory: 2048
app_uris: 4
services: 16
apps: 20
<% else %>
- memory: <%= properties.cc.default_account_capacity.memory || 2048 %>
- app_uris: <%= properties.cc.default_account_capacity.app_uris || 4 %>
- services: <%= properties.cc.default_account_capacity.services || 16 %>
- apps: <%= properties.cc.default_account_capacity.apps || 16 %>
+ memory: <%= cc_props.default_account_capacity.memory || 2048 %>
+ app_uris: <%= cc_props.default_account_capacity.app_uris || 4 %>
+ services: <%= cc_props.default_account_capacity.services || 16 %>
+ apps: <%= cc_props.default_account_capacity.apps || 16 %>
<% end %>
builtin_services:
@@ -186,8 +193,8 @@ service_proxy:
token: <%= "['#{service_proxy_tokens.join("' , '")}']" %>
<%
-if properties.cc.service_extension && properties.cc.service_extension.service_lifecycle
- max_upload_size = properties.cc.service_extension.service_lifecycle.max_upload_size || 1
+if cc_props.service_extension && cc_props.service_extension.service_lifecycle
+ max_upload_size = cc_props.service_extension.service_lifecycle.max_upload_size || 1
if properties.service_lifecycle
if properties.service_lifecycle.serialization_max_download_size
max_download_size = properties.service_lifecycle.serialization_max_download_size
View
6 jobs/cloud_controller_ng/monit
@@ -3,3 +3,9 @@ check process cloud_controller_ng
start program "/var/vcap/jobs/cloud_controller_ng/bin/cloud_controller_ng_ctl start"
stop program "/var/vcap/jobs/cloud_controller_ng/bin/cloud_controller_ng_ctl stop"
group vcap
+
+check process nginx
+ with pidfile /var/vcap/sys/run/nginx/nginx.pid
+ start program "/var/vcap/jobs/cloud_controller_ng/bin/nginx_ctl start"
+ stop program "/var/vcap/jobs/cloud_controller_ng/bin/nginx_ctl stop"
+ group vcap
View
26 jobs/cloud_controller_ng/spec
@@ -1,23 +1,33 @@
---
name: cloud_controller_ng
templates:
+ nginx_ctl: bin/nginx_ctl
+ nginx.conf.erb: config/nginx.conf
+ mime.types : config/mime.types
cloud_controller_ng.yml.erb: config/cloud_controller_ng.yml
cloud_controller_ng_ctl.erb: bin/cloud_controller_ng_ctl
syslog_forwarder.conf.erb: config/syslog_forwarder.conf
iptables.conf.erb: config/iptables.conf
+ nfs-common: config/nfs-common
+ idmapd.conf.erb: config/idmapd.conf
+ node_staging.yml: config/staging/node.yml
+ sinatra_staging.yml: config/staging/sinatra.yml
+ java_web_staging.yml: config/staging/java_web.yml
+ spring_staging.yml: config/staging/spring.yml
+ rails3_staging.yml: config/staging/rails3.yml
+ grails_staging.yml: config/staging/grails.yml
+ lift_staging.yml: config/staging/lift.yml
+ rack_staging.yml: config/staging/rack.yml
+ standalone_staging.yml: config/staging/standalone.yml
+ play_staging.yml: config/staging/play.yml
+ runtimes.yml: config/runtimes.yml
packages:
- common
- cloud_controller_ng
+ - nginx
- libpq
- sqlite
- - ruby
- - dea_node04
- - dea_node06
- - dea_node08
- - dea_ruby18
- - dea_ruby19
- - dea_jvm
- - dea_jvm7
+ - ruby_next
- dea_transition
- dea_seed
- syslog_aggregator
View
237 jobs/cloud_controller_ng/templates/cloud_controller_ng.yml.erb
@@ -1,11 +1,31 @@
---
-port: 8181
+local_route: <%= spec.networks.send(properties.networks.apps).ip %>
+port: 9022 # external CC port
pid_filename: /var/vcap/sys/run/cloud_controller_ng/cloud_controller_ng.pid
nats_uri: nats://<%= properties.nats.user %>:<%= properties.nats.password %>@<%= properties.nats.address %>:<%= properties.nats.port %>
-external_domain: api2.<%= properties.domain %>
+external_domain: <%= properties.ccng.external_host || "api2" %>.<%= properties.domain %>
-bootstrap_admin_email: sre@vmware.com
+<% system_domains = properties.ccng.system_domains || [ properties.domain ] %>
+system_domains: [ <%= system_domains.join(", ") %> ]
+
+cc_partition: <%= properties.ccng.cc_partition || "ng" %>
+
+bootstrap_admin_email: <%= properties.ccng.bootstrap_admin_email || "sre@vmware.com" %>
+
+bulk_api:
+ auth_user: <%= properties.ccng.bulk_api_user || "bulk_api" %>
+ auth_password: <%= properties.ccng.bulk_api_password %>
+
+service_lifecycle:
+ max_upload_size: 1
+ upload_token: <%= properties.serialization_data_server.upload_token %>
+ upload_timeout: <%= properties.serialization_data_server.upload_timeout %>
+ serialization_data_server: <%= properties.service_lifecycle.serialization_data_server.map {|ip| "http://#{ip}:#{properties.serialization_data_server.port}" } %>
+
+nginx:
+ use_nginx: <%= properties.ccng.use_nginx || true %>
+ instance_socket: "/var/vcap/sys/run/cloud_controller_ng/cloud_controller.sock"
info:
name: <%= properties.name || "vcap" %>
@@ -20,10 +40,14 @@ info:
directories:
<% if properties.directories %>
droplets: <%= properties.directories.droplets %>
+ resources: <%= properties.directories.resources %>
+ staging_manifests: <%= properties.directories.staging_manifests %>
tmpdir: <%= properties.directories.tmpdir %>
<% else %>
droplets: "/var/vcap/shared/droplets"
- tmpdir: " /var/vcap/data/cloud_controller/tmp"
+ resources: "/var/vcap/shared/resources"
+ staging_manifests: "/var/vcap/jobs/cloud_controller_ng/config/staging"
+ tmpdir: "/var/vcap/data/cloud_controller_ng/tmp"
<% end %>
logging:
@@ -31,7 +55,8 @@ logging:
<% if properties.syslog_aggregator %>
syslog: vcap.cloud_controller_ng
<% end %>
- level: debug2
+ level: <%= properties.ccng.logging_level || "info" %>
+ max_retries: <%= properties.ccng.logging_max_retries || 1 %>
<% db = properties.ccdb_ng.databases.find { |db| db.tag == "cc" } %>
<% db_role = properties.ccdb_ng.roles.find { |role| role.tag == "admin" } %>
@@ -39,6 +64,7 @@ db:
database: postgres://<%= db_role.name %>:<%= db_role.password %>@<%= properties.ccdb_ng.address %>:<%= properties.ccdb_ng.port %>/<%= db.name %>
max_connections: <%= properties.ccdb_ng.max_connections || 32 %>
pool_timeout: <%= properties.ccdb_ng.pool_timeout || 10 %>
+ log_level: <%= properties.ccng.db_logging_level || "info" %>
<% if properties.vcap_redis %>
redis:
@@ -49,207 +75,14 @@ redis:
uaa:
url: http://uaa.<%= properties.domain %>
- resource_id: <%= properties.cc.uaa.resource_id %>
+ resource_id: <%= properties.ccng.uaa_resource_id %>
symmetric_secret: <%= properties.uaa.cc.token_secret %>
-quota_manager:
-<% if properties.quota_manager %>
- policy: <%= properties.quota_manager.policy %>
- base_url: <%= properties.quota_manager.base_url %>
- auth_token: <%= properties.quota_manager.auth_token %>
- http_timeout_sec: <%= properties.quota_manager.http_timeout_sec %>
-<% else %>
- policy: "BlindApproval"
- base_url: ""
- auth_token: ""
- http_timeout_sec: 2
-<% end %>
-
# App staging parameters
staging:
max_staging_runtime: <%= properties.cc.max_staging_runtime || 120 %>
auth:
- user: <%= properties.cc.staging_upload_user %>
- password: <%= properties.cc.staging_upload_password %>
-
-# TODO: this will be removed when we do full runtime/framework
-# db population when we work on real running app support
-#
-# The first entry in the runtimes array is the default runtime for that
-# framework.
-
-legacy_framework_manifest:
- django:
- name: django
- runtimes:
- - name: python2
- version: 2.6.5
- description: Python 2.6.5
- appservers: []
-
- detection:
- - "*.py": .
- grails:
- name: grails
- runtimes:
- - name: java
- version: "1.6"
- description: Java 6
- appservers:
- - name: tomcat
- description: Tomcat
- detection:
- - "*.war": true
- java_web:
- name: java_web
- runtimes:
- - name: java
- version: "1.6"
- description: Java 6
- appservers:
- - name: tomcat
- description: Tomcat
- detection:
- - "*.war": true
- lift:
- name: lift
- runtimes:
- - name: java
- version: "1.6"
- description: Java 6
- appservers:
- - name: tomcat
- description: Tomcat
- detection:
- - "*.war": true
- node:
- name: node
- runtimes:
- - name: node06
- version: 0.6.8
- description: Node.js
- - name: node
- version: 0.4.12
- description: Node.js
- appservers: []
-
- detection:
- - "*.js": .
- otp_rebar:
- name: otp_rebar
- runtimes:
- - name: erlangR14B02
- version: R14B02
- description: Erlang R14B02
- appservers: []
-
- detection:
- - releases/*/*.rel: .
- php:
- name: php
- runtimes:
- - name: php
- version: "5.3"
- description: PHP 5
- appservers:
- - name: apache
- description: Apache
- detection:
- - "*.php": true
- rack:
- name: rack
- runtimes:
- - name: ruby18
- version: 1.8.7
- description: Ruby 1.8.7
- - name: ruby19
- version: 1.9.2p180
- description: Ruby 1.9.2
- appservers:
- - name: thin
- description: Thin
- detection:
- - config.ru: true
- - config/environment.rb: false
- rails3:
- name: rails3
- runtimes:
- - name: ruby18
- version: 1.8.7
- description: Ruby 1.8.7
- - name: ruby19
- version: 1.9.2p180
- description: Ruby 1.9.2
- appservers:
- - name: thin
- description: Thin
- detection:
- - config/application.rb: true
- - config/environment.rb: true
- sinatra:
- name: sinatra
- runtimes:
- - name: ruby18
- version: 1.8.7
- description: Ruby 1.8.7
- - name: ruby19
- version: 1.9.2p180
- description: Ruby 1.9.2
- appservers:
- - name: thin
- description: Thin
- detection:
- - "*.rb": require 'sinatra'|require "sinatra"
- - config.ru: false
- - config/environment.rb: false
- spring:
- name: spring
- runtimes:
- - name: java
- version: "1.6"
- description: Java 6
- appservers:
- - name: tomcat
- description: Tomcat
- detection:
- - "*.war": true
- standalone:
- name: standalone
- runtimes:
- - name: ruby18
- version: 1.8.7
- description: Ruby 1.8.7
- - name: ruby19
- version: 1.9.2p180
- description: Ruby 1.9.2
- - name: python2
- version: 2.6.5
- description: Python 2.6.5
- - name: java
- version: "1.6"
- description: Java 6
- - name: node
- version: 0.4.12
- description: Node.js
- - name: node06
- version: 0.6.8
- description: Node.js
- - name: erlangR14B02
- version: R14B02
- description: Erlang R14B02
- - name: php
- version: "5.3"
- description: PHP 5
- appservers: []
-
- detection:
- wsgi:
- name: wsgi
- runtimes:
- - name: python2
- version: 2.6.5
- description: Python 2.6.5
- appservers: []
+ user: <%= properties.ccng.staging_upload_user %>
+ password: <%= properties.ccng.staging_upload_password %>
- detection:
- - "*.py": .
+runtimes_file: /var/vcap/jobs/cloud_controller_ng/config/runtimes.yml
View
23 jobs/cloud_controller_ng/templates/cloud_controller_ng_ctl.erb
@@ -1,7 +1,7 @@
#!/bin/bash
export CC_JOB_DIR=/var/vcap/jobs/cloud_controller_ng
-export PATH=/var/vcap/packages/ruby/bin:$PATH
+export PATH=/var/vcap/packages/ruby_next/bin:$PATH
CC_PACKAGE_DIR=/var/vcap/packages/cloud_controller_ng
NFS_SHARE=/var/vcap/shared
@@ -64,16 +64,25 @@ case $1 in
/etc/init.d/idmapd restart
if grep -qs $NFS_SHARE /proc/mounts; then
- echo "Found NFS mount"
- else
- echo "Mounting NFS..."
- mount -o timeo=10,intr,lookupcache=positive -t nfs4 <%= properties.nfs_server.address %>:<%= properties.nfs_server.share || "/" %> $NFS_SHARE
- if [ $? != 0 ]; then
- echo "Cannot mount NFS, exiting..."
+ echo "Found NFS mount, unmounting..."
+ umount $NFS_SHARE
+ if [ $? -ne 0 ]; then
+ echo "Failed to unmount NFS, exiting..."
exit 1
fi
fi
+ echo "Mounting NFS..."
+ <% if properties.nfs_server.nfsv4 %>
+ mount -o timeo=10,intr,lookupcache=positive -t nfs4 <%= properties.nfs_server.address %>:<%= properties.nfs_server.share || "/" %> $NFS_SHARE
+ <% else %>
+ mount -o timeo=10,intr,lookupcache=positive -t nfs <%= properties.nfs_server.address %>:<%= properties.nfs_server.share || "/var/vcap/store/shared" %> $NFS_SHARE
+ <% end %>
+ if [ $? != 0 ]; then
+ echo "Cannot mount NFS, exiting..."
+ exit 1
+ fi
+
chpst -u vcap:vcap touch $NFS_SHARE/.nfs_test
if [ $? != 0 ]; then
View
10 jobs/cloud_controller_ng/templates/grails_staging.yml
@@ -0,0 +1,10 @@
+---
+name: grails
+
+runtimes:
+ - java:
+ default: true
+ - java7:
+ default: false
+detection:
+ - *.war: true # TODO - this thinks everything with an XML file is Spring
View
10 jobs/cloud_controller_ng/templates/idmapd.conf.erb
@@ -0,0 +1,10 @@
+[General]
+
+Verbosity = 0
+Pipefs-Directory = /var/lib/nfs/rpc_pipefs
+Domain = <%= properties.nfs_server && properties.nfs_server.idmapd_domain ? properties.nfs_server.idmapd_domain : "localdomain" %>
+
+[Mapping]
+
+Nobody-User = nobody
+Nobody-Group = nogroup
View
10 jobs/cloud_controller_ng/templates/java_web_staging.yml
@@ -0,0 +1,10 @@
+---
+name: java_web
+
+runtimes:
+ - java:
+ default: true
+ - java7:
+ default: false
+detection:
+ - *.war: true
View
10 jobs/cloud_controller_ng/templates/lift_staging.yml
@@ -0,0 +1,10 @@
+---
+name: lift
+
+runtimes:
+ - java:
+ default: true
+ - java7:
+ default: false
+detection:
+ - *.war: true
View
73 jobs/cloud_controller_ng/templates/mime.types
@@ -0,0 +1,73 @@
+types {
+ text/html html htm shtml;
+ text/css css;
+ text/xml xml;
+ image/gif gif;
+ image/jpeg jpeg jpg;
+ application/x-javascript js;
+ application/atom+xml atom;
+ application/rss+xml rss;
+
+ text/mathml mml;
+ text/plain txt;
+ text/vnd.sun.j2me.app-descriptor jad;
+ text/vnd.wap.wml wml;
+ text/x-component htc;
+
+ image/png png;
+ image/tiff tif tiff;
+ image/vnd.wap.wbmp wbmp;
+ image/x-icon ico;
+ image/x-jng jng;
+ image/x-ms-bmp bmp;
+ image/svg+xml svg;
+
+ application/java-archive jar war ear;
+ application/mac-binhex40 hqx;
+ application/msword doc;
+ application/pdf pdf;
+ application/postscript ps eps ai;
+ application/rtf rtf;
+ application/vnd.ms-excel xls;
+ application/vnd.ms-powerpoint ppt;
+ application/vnd.wap.wmlc wmlc;
+ application/vnd.wap.xhtml+xml xhtml;
+ application/vnd.google-earth.kml+xml kml;
+ application/vnd.google-earth.kmz kmz;
+ application/x-7z-compressed 7z;
+ application/x-cocoa cco;
+ application/x-java-archive-diff jardiff;
+ application/x-java-jnlp-file jnlp;
+ application/x-makeself run;
+ application/x-perl pl pm;
+ application/x-pilot prc pdb;
+ application/x-rar-compressed rar;
+ application/x-redhat-package-manager rpm;
+ application/x-sea sea;
+ application/x-shockwave-flash swf;
+ application/x-stuffit sit;
+ application/x-tcl tcl tk;
+ application/x-x509-ca-cert der pem crt;
+ application/x-xpinstall xpi;
+ application/zip zip;
+
+ application/octet-stream bin exe dll;
+ application/octet-stream deb;
+ application/octet-stream dmg;
+ application/octet-stream eot;
+ application/octet-stream iso img;
+ application/octet-stream msi msp msm;
+
+ audio/midi mid midi kar;
+ audio/mpeg mp3;
+ audio/x-realaudio ra;
+
+ video/3gpp 3gpp 3gp;
+ video/mpeg mpeg mpg;
+ video/quicktime mov;
+ video/x-flv flv;
+ video/x-mng mng;
+ video/x-ms-asf asx asf;
+ video/x-ms-wmv wmv;
+ video/x-msvideo avi;
+}
View
2  jobs/cloud_controller_ng/templates/nfs-common
@@ -0,0 +1,2 @@
+NEED_IDMAPD=yes
+NEED_GSSD=no
View
129 jobs/cloud_controller_ng/templates/nginx.conf.erb
@@ -0,0 +1,129 @@
+# deployment cloudcontroller nginx.conf
+user vcap vcap;
+
+error_log /var/vcap/sys/log/nginx/nginx.error.log;
+pid /var/vcap/sys/run/nginx/nginx.pid;
+
+events {
+ worker_connections 8192;
+ use epoll;
+}
+
+http {
+ include mime.types;
+ default_type text/html;
+ server_tokens off;
+
+ log_format main '$host - [$time_local] '
+ '"$request" $status $bytes_sent '
+ '"$http_referer" "$http_user_agent" '
+ '$proxy_add_x_forwarded_for response_time:$upstream_response_time';
+
+ access_log /var/vcap/sys/log/nginx/nginx.access.log main;
+
+ sendfile on; #enable use of sendfile()
+ tcp_nopush on;
+ tcp_nodelay on; #disable nagel's algorithm
+
+ keepalive_timeout 75 20; #inherited from router
+
+ client_max_body_size 256M; #already enforced upstream/but doesn't hurt.
+
+ upstream cloud_controller {
+ server unix:/var/vcap/sys/run/cloud_controller_ng/cloud_controller.sock;
+ }
+
+ server {
+ listen 9022;
+ server_name _;
+ server_name_in_redirect off;
+
+ # proxy and log all CC traffic
+ location / {
+ access_log /var/vcap/sys/log/nginx/nginx.access.log main;
+ proxy_buffering off;
+ proxy_set_header Host $host;
+ proxy_set_header X-Real_IP $remote_addr;
+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+ proxy_redirect off;
+ proxy_connect_timeout 10;
+ proxy_send_timeout <%= properties.router.client_inactivity_timeout || 30 %>;
+ proxy_read_timeout <%= properties.router.app_inactivity_timeout || 30 %>;
+ proxy_pass http://cloud_controller;
+ }
+
+ # used for x-accel-redirect uri://location/foo.txt
+ # nginx will serve the file root || location || foo.txt
+ location /droplets/ {
+ internal;
+ root /var/vcap/shared;
+ }
+
+ location ~ (/apps/.*/application|/v2/apps/.*/bits|/services/v\d+/configurations/.*/serialized/data) {
+ # Pass altered request body to this location
+ upload_pass @cc_uploads;
+ upload_pass_args on;
+
+ # Store files to this directory
+ upload_store /var/vcap/data/cloud_controller_ng/tmp/uploads;
+
+ # No limit for output body forwarded to CC
+ upload_max_output_body_len 0;
+
+ # Allow uploaded files to be read only by user
+ upload_store_access user:r;
+
+ # Set specified fields in request body
+ upload_set_form_field "${upload_field_name}_name" $upload_file_name;
+ upload_set_form_field "${upload_field_name}_path" $upload_tmp_path;
+
+ #forward the following fields from existing body
+ upload_pass_form_field "^resources$";
+ upload_pass_form_field "^_method$";
+
+ #on any error, delete uploaded files.
+ upload_cleanup 400-505;
+ }
+
+ # Droplet uploads from the stager should be authenticated
+ location /staging/droplets/ {
+ # Pass along auth header
+ set $auth_header $upstream_http_x_auth;
+ proxy_set_header Authorization $auth_header;
+
+ # Pass altered request body to this location
+ upload_pass @cc_uploads;
+
+ # Store files to this directory
+ upload_store /var/vcap/data/cloud_controller_ng/tmp/staged_droplet_uploads;
+
+ # Allow uploaded files to be read only by user
+ upload_store_access user:r;
+
+ # Set specified fields in request body
+ upload_set_form_field "droplet_path" $upload_tmp_path;
+
+ #on any error, delete uploaded files.
+ upload_cleanup 400-505;
+ }
+
+ # Pass altered request body to a backend
+ location @cc_uploads {
+ proxy_pass http://unix:/var/vcap/sys/run/cloud_controller_ng/cloud_controller.sock;
+ }
+
+ location ~ ^/internal_redirect/(.*){
+ # only allow internal redirects
+ internal;
+
+ set $download_url $1;
+
+ #have to manualy pass along auth header
+ set $auth_header $upstream_http_x_auth;
+ proxy_set_header Authorization $auth_header;
+
+ # Download the file and send it to client
+ proxy_pass $download_url;
+ }
+ }
+}
View
47 jobs/cloud_controller_ng/templates/nginx_ctl
@@ -0,0 +1,47 @@
+#!/bin/bash
+
+RUN_DIR=/var/vcap/sys/run/nginx
+LOG_DIR=/var/vcap/sys/log/nginx
+UPLOADS_DIR=/var/vcap/data/cloud_controller_ng/tmp/uploads
+STAGED_UPLOADS_DIR=/var/vcap/data/cloud_controller_ng/tmp/staged_droplet_uploads
+PIDFILE=$RUN_DIR/nginx.pid
+CC_PACKAGE_DIR=/var/vcap/packages/cloud_controller_ng
+CC_JOB_DIR=/var/vcap/jobs/cloud_controller_ng
+
+source /var/vcap/packages/common/utils.sh
+
+case $1 in
+
+ start)
+ pid_guard $PIDFILE "Nginx"
+
+ mkdir -p $RUN_DIR
+ mkdir -p $LOG_DIR
+
+ echo $$ > $PIDFILE
+
+ if [ ! -d "$UPLOADS_DIR" ]; then
+ chpst -u vcap:vcap mkdir -p $UPLOADS_DIR
+ fi
+
+ if [ ! -d "$STAGED_UPLOADS_DIR" ]; then
+ chpst -u vcap:vcap mkdir -p $STAGED_UPLOADS_DIR
+ fi
+
+ exec /var/vcap/packages/nginx/sbin/nginx \
+ -c $CC_JOB_DIR/config/nginx.conf \
+ >>$LOG_DIR/nginx.stdout.log \
+ 2>>$LOG_DIR/nginx.stderr.log
+ ;;
+
+ stop)
+ kill_and_wait $PIDFILE
+
+ ;;
+
+ *)
+ echo "Usage: nginx_ctl {start|stop}"
+
+ ;;
+
+esac
View
11 jobs/cloud_controller_ng/templates/node_staging.yml
@@ -0,0 +1,11 @@
+---
+name: "node"
+runtimes:
+ - node:
+ default: true
+ - node06:
+ default: false
+ - node08:
+ default: false
+detection:
+ - "*.js": '.'
View
10 jobs/cloud_controller_ng/templates/play_staging.yml
@@ -0,0 +1,10 @@
+---
+name: play
+
+runtimes:
+ - java:
+ default: true
+ - java7:
+ default: false
+detection:
+ - "lib/play.*.jar": true
View
12 jobs/cloud_controller_ng/templates/rack_staging.yml
@@ -0,0 +1,12 @@
+---
+name: "rack"
+runtimes:
+ - ruby18:
+ default: true
+ - ruby19:
+ default: false
+detection:
+ - "config.ru": true
+ - "config/environment.rb": false # and config/environment.rb must not exist
+
+# vim: filetype=yaml
View
11 jobs/cloud_controller_ng/templates/rails3_staging.yml
@@ -0,0 +1,11 @@
+---
+name: rails3
+
+runtimes:
+ - ruby18:
+ default: true
+ - ruby19:
+ default: false
+detection:
+ - config/application.rb: true
+ - config/environment.rb: true
View
150 jobs/cloud_controller_ng/templates/runtimes.yml
@@ -0,0 +1,150 @@
+---
+java:
+ description: Java 6
+ version: "1.6.0_24"
+ executable: /var/vcap/packages/dea_jvm/bin/java
+ staging: /var/vcap/packages/ruby/bin/ruby stage
+ version_output: 1.6.0_24
+ version_flag: '-version'
+ environment:
+ PATH: /var/vcap/packages/ruby/bin:/var/vcap/packages/dea_jvm/bin:/var/vcap/packages/imagemagick/bin:$PATH
+ JAVA_HOME: /var/vcap/packages/dea_jvm
+ LD_LIBRARY_PATH: '/var/vcap/packages/mysqlclient/lib/mysql:/var/vcap/packages/sqlite/lib:/var/vcap/packages/libpq/lib:/var/vcap/packages/imagemagick/lib:$LD_LIBRARY_PATH'
+ debug_modes:
+ - run
+ - suspend
+ debug_env:
+ run:
+ - JAVA_OPTS="$JAVA_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,address=$VCAP_DEBUG_PORT,server=y,suspend=n"
+ suspend:
+ - JAVA_OPTS="$JAVA_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,address=$VCAP_DEBUG_PORT,server=y,suspend=y"
+ status:
+ name: current
+ series: java
+ category: java
+java7:
+ description: Java 7
+ version: "1.7.0_04"
+ executable: /var/vcap/packages/dea_jvm7/bin/java
+ staging: /var/vcap/packages/ruby/bin/ruby stage
+ version_output: 1.7.0_04
+ version_flag: '-version'
+ environment:
+ PATH: /var/vcap/packages/ruby/bin:/var/vcap/packages/dea_jvm7/bin:/var/vcap/packages/imagemagick/bin:$PATH
+ JAVA_HOME: /var/vcap/packages/dea_jvm7
+ LD_LIBRARY_PATH: '/var/vcap/packages/mysqlclient/lib/mysql:/var/vcap/packages/sqlite/lib:/var/vcap/packages/libpq/lib:/var/vcap/packages/imagemagick/lib:$LIBRARY_PATH'
+ debug_modes:
+ - run
+ - suspend
+ debug_env:
+ run:
+ - JAVA_OPTS="$JAVA_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,address=$VCAP_DEBUG_PORT,server=y,suspend=n"
+ suspend:
+ - JAVA_OPTS="$JAVA_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,address=$VCAP_DEBUG_PORT,server=y,suspend=y"
+ status:
+ name: current
+ series: java7
+ category: java
+node:
+ version: "0.4.12"
+ description: "Node.js"
+ executable: /var/vcap/packages/dea_node04/bin/node
+ npm: /var/vcap/packages/dea_node04/npm/bin/npm-cli.js
+ staging: /var/vcap/packages/ruby/bin/ruby stage
+ version_output: 0.4.12
+ version_flag: '-v'
+ environment:
+ LD_LIBRARY_PATH: '/var/vcap/packages/mysqlclient/lib/mysql:/var/vcap/packages/sqlite/lib:/var/vcap/packages/libpq/lib:/var/vcap/packages/imagemagick/lib:$LD_LIBRARY_PATH'
+ PATH: /var/vcap/packages/imagemagick/bin:/var/vcap/packages/dea_node04/bin:$PATH
+ debug_modes:
+ - run
+ - suspend
+ debug_env:
+ run:
+ - NODE_ARGS="--debug=$VCAP_DEBUG_PORT"
+ suspend:
+ - NODE_ARGS="--debug-brk=$VCAP_DEBUG_PORT"
+ status:
+ name: current
+ series: node
+ category: node
+node06:
+ version: "0.6.8"
+ description: "Node.js"
+ executable: /var/vcap/packages/dea_node06/bin/node
+ npm: /var/vcap/packages/dea_node06/bin/npm
+ staging: /var/vcap/packages/ruby/bin/ruby stage
+ version_output: 0.6.8
+ version_flag: '-v'
+ environment:
+ LD_LIBRARY_PATH: '/var/vcap/packages/mysqlclient/lib/mysql:/var/vcap/packages/sqlite/lib:/var/vcap/packages/libpq/lib:/var/vcap/packages/imagemagick/lib:$LD_LIBRARY_PATH'
+ PATH: /var/vcap/packages/imagemagick/bin:/var/vcap/packages/dea_node06/bin:$PATH
+ debug_modes:
+ - run
+ - suspend
+ debug_env:
+ run:
+ - NODE_ARGS="--debug=$VCAP_DEBUG_PORT"
+ suspend:
+ - NODE_ARGS="--debug-brk=$VCAP_DEBUG_PORT"
+ status:
+ name: current
+ series: node06
+ category: node
+node08:
+ version: "0.8.2"
+ description: "Node.js"
+ executable: /var/vcap/packages/dea_node08/bin/node
+ npm: /var/vcap/packages/dea_node08/bin/npm
+ staging: /var/vcap/packages/ruby/bin/ruby stage
+ version_output: 0.8.2
+ version_flag: '-v'
+ environment:
+ PATH: /var/vcap/packages/imagemagick/bin:/var/vcap/packages/dea_node08/bin:$PATH
+ LD_LIBRARY_PATH: '/var/vcap/packages/mysqlclient/lib/mysql:/var/vcap/packages/sqlite/lib:/var/vcap/packages/libpq/lib:/var/vcap/packages/imagemagick/lib:$LD_LIBRARY_PATH'
+ debug_modes:
+ - run
+ - suspend
+ debug_env:
+ run:
+ - NODE_ARGS="--debug=$VCAP_DEBUG_PORT"
+ suspend:
+ - NODE_ARGS="--debug-brk=$VCAP_DEBUG_PORT"
+ status:
+ name: current
+ series: node08
+ category: node
+ruby18:
+ description: Ruby 1.8
+ version: "1.8.7p357"
+ executable: /var/vcap/packages/dea_ruby18/bin/ruby
+ staging: /var/vcap/packages/ruby/bin/ruby stage
+ version_output: 1.8.7
+ version_flag: "-e 'puts RUBY_VERSION'"
+ additional_checks: "-e 'puts RUBY_PATCHLEVEL == 357'"
+ environment:
+ LD_LIBRARY_PATH: '/var/vcap/packages/mysqlclient/lib/mysql:/var/vcap/packages/sqlite/lib:/var/vcap/packages/libpq/lib:/var/vcap/packages/imagemagick/lib:$LD_LIBRARY_PATH'
+ BUNDLE_GEMFILE:
+ PATH: /var/vcap/packages/imagemagick/bin:/var/vcap/packages/dea_transition/rubygems/1.8/bin:/var/vcap/packages/dea_ruby18/bin:/var/vcap/packages/dea_node08/bin:$PATH
+ GEM_PATH: /var/vcap/packages/dea_transition/rubygems/1.8:$GEM_PATH
+ status:
+ name: current
+ series: ruby18
+ category: ruby
+ruby19:
+ description: Ruby 1.9
+ version: "1.9.2p180"
+ executable: /var/vcap/packages/dea_ruby19/bin/ruby
+ staging: /var/vcap/packages/ruby/bin/ruby stage
+ version_output: 1.9.2
+ version_flag: "-e 'puts RUBY_VERSION'"
+ additional_checks: "-e 'puts RUBY_PATCHLEVEL == 180'"
+ environment:
+ LD_LIBRARY_PATH: '/var/vcap/packages/mysqlclient/lib/mysql:/var/vcap/packages/sqlite/lib:/var/vcap/packages/libpq/lib:/var/vcap/packages/imagemagick/lib:$LD_LIBRARY_PATH'
+ BUNDLE_GEMFILE:
+ PATH: /var/vcap/packages/imagemagick/bin:/var/vcap/packages/dea_transition/rubygems/1.9.1/bin:/var/vcap/packages/dea_ruby19/bin:/var/vcap/packages/dea_node08/bin:$PATH
+ GEM_PATH: /var/vcap/packages/dea_transition/rubygems/1.9.1:$GEM_PATH
+ status:
+ name: current
+ series: ruby19
+ category: ruby
View
12 jobs/cloud_controller_ng/templates/sinatra_staging.yml
@@ -0,0 +1,12 @@
+---
+name: sinatra
+
+runtimes:
+ - ruby18:
+ default: true
+ - ruby19:
+ default: false
+
+detection:
+ - "*.rb": "\\s*require[\\s\\(]*['\"]sinatra(/base)?['\"]" # .rb files in the root dir containing a require?
+ - config/environment.rb: false # and config/environment.rb must not exist
View
10 jobs/cloud_controller_ng/templates/spring_staging.yml
@@ -0,0 +1,10 @@
+---
+name: spring
+
+runtimes:
+ - java:
+ default: true
+ - java7:
+ default: false
+detection:
+ - *.war: true # TODO - this thinks everything with an XML file is Spring
View
18 jobs/cloud_controller_ng/templates/standalone_staging.yml
@@ -0,0 +1,18 @@
+---
+name: standalone
+
+runtimes:
+ - java:
+ default: false
+ - java7:
+ default: false
+ - ruby18:
+ default: false
+ - ruby19:
+ default: false
+ - node:
+ default: false
+ - node06:
+ default: false
+ - node08:
+ default: false
View
2  jobs/dea_next/spec
@@ -15,7 +15,7 @@ packages:
- common
- dea_next
- warden
-- ruby
+- ruby_next
- golang
- sqlite
- mysqlclient
View
4 jobs/dea_next/templates/dea_ctl
@@ -1,6 +1,6 @@
#!/bin/bash
-export PATH=/var/vcap/packages/ruby/bin:$PATH
+export PATH=/var/vcap/packages/ruby_next/bin:$PATH
RUN_DIR=/var/vcap/sys/run/dea_next
LOG_DIR=/var/vcap/sys/log/dea_next
PIDFILE=$RUN_DIR/dea_next.pid
@@ -51,7 +51,7 @@ case $1 in
/var/vcap/packages/syslog_aggregator/setup_syslog_forwarder.sh $JOB_DIR/config
<% end %>
- exec /var/vcap/packages/ruby/bin/ruby /var/vcap/packages/dea_next/bin/dea \
+ exec /var/vcap/packages/ruby_next/bin/ruby /var/vcap/packages/dea_next/bin/dea \
$JOB_DIR/config/dea.yml \
>>$LOG_DIR/dea.stdout.log \
2>>$LOG_DIR/dea.stderr.log
View
6 jobs/dea_next/templates/warden_ctl
@@ -55,12 +55,12 @@ case $1 in
# Clear existing containers if warden was not drained
if [ ! -f /var/vcap/data/warden/depot/drained ]
then
- root/linux/clear.sh /var/vcap/data/warden/depot
+ find /var/vcap/data/warden/depot/ -mindepth 1 -maxdepth 1 -type d | xargs -t -n1 root/linux/destroy.sh
fi
- export PATH=/var/vcap/packages/ruby/bin:$PATH
+ export PATH=/var/vcap/packages/ruby_next/bin:$PATH
- exec /var/vcap/packages/ruby/bin/bundle exec \
+ exec /var/vcap/packages/ruby_next/bin/bundle exec \
rake warden:start[$WARDEN_CONF_DIR/warden.yml] \
1>>$LOG_DIR/warden.stdout.log \
2>>$LOG_DIR/warden.stderr.log
View
23 jobs/health_manager/templates/health_manager.yml.erb
@@ -1,6 +1,12 @@
---
-<% db = properties.ccdb.databases.find { |db| db.tag == "cc" } %>
-<% db_role = properties.ccdb.roles.find { |role| role.tag == "admin" } %>
+<% hm_props_name = properties.hm_props || 'health_manager' %>
+<% hm_props = properties.send(hm_props_name) || OpenStruct.new %>
+
+<% ccdb_prop_name = properties.db || 'ccdb' %>
+<% ccdb = properties.send(ccdb_prop_name) %>
+<% db_name = ccdb.databases.find { |db| db.tag == "cc" }.name %>
+<% db_role = ccdb.roles.find { |role| role.tag == "admin" } %>
+
<% network = spec.networks.send(properties.networks.apps) %>
<% nats = properties.nats %>
@@ -15,20 +21,21 @@ logging:
pid: /var/vcap/sys/run/health_manager/health_manager.pid
rails_environment: production
index: <%= spec.index %>
+cc_partition: <%= hm_props.cc_partition || "default" %>
database_environment:
production:
- database: <%= db.name %>
- host: <%= properties.ccdb.address %>
- port: <%= properties.ccdb.port %>
+ database: <%= db_name %>
+ host: <%= ccdb.address %>
+ port: <%= ccdb.port %>
encoding: utf8
username: <%= db_role.name %>
password: <%= db_role.password %>
- pool: <%= properties.ccdb.pool_size %>
+ pool: <%= ccdb.pool_size %>
timeout: 2000
template: template0 # Required for utf8 encoding
adapter: postgresql
intervals:
- <% intervals = properties.health_manager && properties.health_manager.intervals || OpenStruct.new %>
+ <% intervals = hm_props.intervals || OpenStruct.new %>
database_scan: <%= intervals.database_scan || 60 %>
droplet_lost: <%= intervals.droplet_lost || 30 %>
droplets_analysis: <%= intervals.droplets_analysis || 20 %>
@@ -42,4 +49,4 @@ intervals:
max_db_reconnect_wait: <%= intervals.max_db_reconnect_wait || 300 %>
inactivity_period_for_spindown: <%= intervals.inactivity_period_for_spindown || -1 %>
-dequeueing_rate: <%= properties.health_manager && properties.health_manager.dequeueing_rate || 50 %>
+dequeueing_rate: <%= hm_props.dequeueing_rate || 50 %>
View
12 jobs/health_manager_next/templates/health_manager_next.yml.erb
@@ -1,4 +1,10 @@
---
+<% cc_props_name = properties.cc_props || 'cc' %>
+<% cc_props = properties.send(cc_props_name) %>
+
+<% hm_props_name = properties.hm_props || 'health_manager' %>
+<% hm_props = properties.send(hm_props_name) || OpenStruct.new %>
+
local_route: <%= spec.networks.send(properties.networks.apps).ip %>
mbus: nats://<%= properties.nats.user %>:<%= properties.nats.password %>@<%= properties.nats.address %>:<%= properties.nats.port %>
logging:
@@ -10,10 +16,10 @@ logging:
pid: /var/vcap/sys/run/health_manager_next/health_manager_next.pid
index: <%= spec.index %>
bulk_api:
- host: <%= properties.cc.srv_api_uri %>
+ host: <%= cc_props.srv_api_uri %>
intervals:
- <% intervals = properties.health_manager && properties.health_manager.intervals || OpenStruct.new %>
+ <% intervals = hm_props.intervals || OpenStruct.new %>
expected_state_update: <%= intervals.expected_state_update || 60 %>
droplet_lost: <%= intervals.droplet_lost || 30 %>
droplets_analysis: <%= intervals.droplets_analysis || 20 %>
@@ -24,4 +30,4 @@ intervals:
max_restart_delay: <%= intervals.max_restart_delay || 480 %>
giveup_crash_number: <%= intervals.giveup_crash_number || 4 %>
stable_state: <%= intervals.stable_state || 60 %>
-dequeueing_rate: <%= properties.health_manager && properties.health_manager.dequeueing_rate || 50 %>
+dequeueing_rate: <%= hm_props.dequeueing_rate || 50 %>
View
8 jobs/nats/templates/nats.yml.erb
@@ -1,11 +1,13 @@
---
+<% nats_props_name = properties.nats_props || "nats" %>
+<% nats_props = properties.send(nats_props_name) || OpenStruct.new %>
net: <%= spec.networks.send(properties.networks.apps).ip %>
-port: <%= properties.nats.port %>
+port: <%= nats_props.port %>
pid_file: /var/vcap/sys/run/nats/nats.pid
log_file: /var/vcap/sys/log/nats/nats.log
authorization:
- user: <%= properties.nats.user %>
- password: <%= properties.nats.password %>
+ user: <%= nats_props.user %>
+ password: <%= nats_props.password %>
timeout: 1
View
9 jobs/postgres/templates/postgres_ctl.erb
@@ -95,6 +95,15 @@ case "$1" in
su - vcap -c "$PACKAGE_DIR/bin/createdb \"<%= database.name %>\" -p $PORT"
set -e
+ <% if database.citext %>
+ echo "Trying to install citext..."
+ set +e
+ $PACKAGE_DIR/bin/psql -U vcap -p $PORT \
+ -d "<%= database.name %>" \
+ -f $PACKAGE_DIR/share/postgresql/contrib/citext.sql
+ set -e
+ <% end %>
+
<% if database.run_on_every_startup %>
<% database.run_on_every_startup.each do |query| %>
$PACKAGE_DIR/bin/psql -U vcap -p $PORT \
View
20 packages/ruby_next/packaging
@@ -0,0 +1,20 @@
+# abort script on any command that exit with a non zero value
+set -e
+
+tar xzf ruby/ruby-1.9.3-p194.tar.gz
+(
+ cd ruby-1.9.3-p194
+ ./configure --prefix=${BOSH_INSTALL_TARGET} --disable-install-doc
+ make
+ make install
+)
+
+tar zxvf ruby/rubygems-1.8.24.tgz
+
+(
+ cd rubygems-1.8.24
+
+ ${BOSH_INSTALL_TARGET}/bin/ruby setup.rb
+)
+
+${BOSH_INSTALL_TARGET}/bin/gem install ruby/bundler-1.1.3.gem --no-ri --no-rdoc
View
6 packages/ruby_next/spec
@@ -0,0 +1,6 @@
+---
+name: ruby_next
+files:
+- ruby/ruby-1.9.3-p194.tar.gz
+- ruby/rubygems-1.8.24.tgz
+- ruby/bundler-1.1.3.gem
2  src/cloud_controller_ng
@@ -1 +1 @@
-Subproject commit c012225d22bf22cad27ed25042598f8cec842ea5
+Subproject commit c810a010bff1d30c35ea88c30b5b6b8bf892a1da
2  src/dea_next
@@ -1 +1 @@
-Subproject commit ed1e551a744582cc1d4f54d81521fdec34a8cc1f
+Subproject commit 136225000663f4f5eb684e21fb68a9674a7b3ad7
2  src/warden
@@ -1 +1 @@
-Subproject commit d8b0a93d6488c2c33954c6ad6f55aa6ccf9a4279
+Subproject commit b5ab5fc241d1d3e45e6d9dd7b9a15e334e2d973f
Please sign in to comment.
Something went wrong with that request. Please try again.