Permalink
Commits on Aug 23, 2011
  1. Merge "Add libpg-dev package to the ruby recipe so pg gem can be inst…

    …alled."
    Patrick Bozeman committed with Gerrit Code Review Aug 23, 2011
Commits on Aug 22, 2011
  1. Merge "Add external_uri support"

    mahpat committed with Gerrit Code Review Aug 22, 2011
  2. Add external_uri support

    Tested my changes with cc and services configured to go against my dev23 setup
    i.e.  "appcloud23.dev.mozycloud.com"
    
    Ran BVTs.
    
    Change-Id: I4e3c97a06c4348c31d610957af6e74cf953ad5c5
    mahpat committed Aug 19, 2011
  3. Merge "Trivial - do not install java if it already exists"

    mahpat committed with Gerrit Code Review Aug 22, 2011
  4. Merge "Cleanup deployment config file handling + some sample multihos…

    …t deployment config files"
    mahpat committed with Gerrit Code Review Aug 22, 2011
  5. Merge "Misc simple bug fixes"

    mahpat committed with Gerrit Code Review Aug 22, 2011
  6. Add libpg-dev package to the ruby recipe so pg gem can be installed.

    Fixes problem where gem complains that the pg_config isn't available.
    
    Change-Id: I1463e2139c4c79fdab8dcc4037bc1ae9386f204e
    oldpatricka committed with Patrick Bozeman Aug 12, 2011
  7. Trivial - do not install java if it already exists

    Fix for issue cf-135. Usually the pakcage command does the right thing but that
    would only work if the same version of java is installed, if openjdk or some
    other flavour is installed the package command fails misreably in this case
    because it cant find the right repo.
    
    Change-Id: I1aef325e557d338141e68dbd91a070b438f62563
    mahpat committed Aug 22, 2011
Commits on Aug 19, 2011
  1. Don't attempt to start the droplet if staging the instance dir fails.

    - Adds logging/error reporting if:
    -- Creating the instance dir fails
    -- Unzipping the droplet fails
    - Sends notification to the HM that the app has crashed so that it
      will ultimately end up marked as flapping.
    
    Test plan:
    
    - BVTs pass locally
    - BVTs pass on my deployment
    - Added unit tests exercising failures to stage the instance dir
    - Verified that HM was notified via:
    -- Deployed the env app locally
    -- Corrupted the droplet "echo foo > /var/vcap/shared/droplets/XXX"
    -- Stopped/Started the app
    -- Verified that the HM received the crash notification
    -- Verified that the app was placed into the flapping state
    
    Change-Id: Iba7cb521fcf9864271bd2b0844e2f1eee5774fde
    mpage committed Aug 19, 2011
  2. Misc simple bug fixes

    bin/vcap - fix a bug where the specified config file was not being used to
    lookup properties.
    
    bin/vcap_dev_setup - check for git clone failures
    
    job_manager.rb - remove postgresql as a services
    
    patch 2 - fix $configdir => $config_dir
    
    Change-Id: Ie0214ce340f62f585f6233a7c7de1e78cc6e1ad8
    mahpat committed Aug 18, 2011
  3. Cleanup deployment config file handling + some sample multihost deplo…

    …yment config files
    
    The deployment config file doesnt need to be an .erb file anymore. Removed some
    old code that was using it as an erb file. Also remove the old config samples in
    deployments/multihost.
    
    patch 3
    Added: sample config files for multihost redis
    
    patch 4
    Remove more old/outdated sample files
    
    patch 5
    Hit myself on the head.(Added white space remover to vi)
    
    Change-Id: I402e5de08eb9a1fa26c6489ca138d5da7229cf14
    mahpat committed Aug 18, 2011
Commits on Aug 18, 2011
  1. Properly close fds for dea children

    This moves the call to close_fds up to immediately before executing
    the shell/su that ultimately spawns the app.
    
    Test plan:
    
    - BVTs pass both locally and on my deployment
    - Verified that DEA children do not have a connection to NATS using lsof
    
    Change-Id: I27b6f86c06bf3476ea527cd8fcf41946c9efcb3e
    mpage committed Aug 18, 2011
Commits on Aug 17, 2011
  1. Multi host part 2 + changes to allow deployment like bash < <(curl vc…

    …ap_dev_setup_url)
    
    Change
      This review has changes related to the following 3 things.
      1. Move all the default config (that was embeded in
          vcap_dev/vcap_dev_setup/job_manager) into chef cookbooks.
      2. Add postgres CCDB.
      3. Multi host setup for services and dea. This meant separating service
      gateway from service nodes. To this effect we added a service gateway role.
    
    Overall a lot of files have changed, most of those changes are related to
    moving all the default config values from the various wrapper scripts to the
    cookbooks, especially the new cookbook called deployment which is the holding
    place for all the deployment related config options.
    
    * dev_setup/bin/vcap:
      Added vcap to dev_setup/bin instead of just changing bin/vcap. The reason for
      this is that opensource CF code will not be in sync with the
      private repo and internal testing of CF dev_setup scrips would not work if it
      relied on the version of vcap in the opensource repo i.e. bin/vcap. So now we
      just package vcap with dev_setup.
    
    * vcap_dev
      * Since all defaults are now maintained in chef scripts. The chef "deployment"
      * role/recipe creates a deployment info file that is consumed by this vcap_dev
      * script.
      * The vcap_dev_setup file now saves the list of components that were installed
        for a deployment. This script only starts the components included in that
        list.
      * Uses the vcap binary from dev_setup/bin
    
    * vcap_dev_setup
      Moved a bunch of directory creating code to the deployment cookbook
    
    * Added a CCDB role/recipe.
      * Creates and Configures the CC postgres database.
    
    * Added deployment cookbook
      * Moves all the comon directory creation code here. Note, we should go back
        and see if we really do need all these various directories and probably come
        up with a well designed directory layout.
    
    Testing:
    * Ran BVTS. Lift BVT failure needs investigating (I will fix/get to the bottom
      of it before i submit this change)
    * Tested the following multi host setup.
      1. Ran mysql database with mysql node on 2 VMs. On a 3rd VM ran everything
      else. Verified that BVTs passed.
      2. Ran dea on 2 different VMs. Ran everything else on a 3rd VM. Verified
      that BVTs passed.
    
        Note: I have only tested the multihost setup with mysql. I will test mongodb
        and redis before submitting this change, I expect those to run without any
        glitches.
    
    Change-Id: I6a084be09a81bf920eebc62be8d7aa6625cc17e9
    mahpat committed Aug 15, 2011
Commits on Aug 12, 2011
  1. Fix for zero length responses being returned

    Change-Id: I3cac7f2985e214f078077a0a5ba660feba306924
    garethr committed with Patrick Bozeman May 26, 2011
  2. Merge "bump services submodule ptr"

    Patrick Bozeman committed with Gerrit Code Review Aug 12, 2011
  3. Merge changes I104e4c78,Ia879da24

    * changes:
      bump test submodule ptr
      bump java submodule ptr
    Patrick Bozeman committed with Gerrit Code Review Aug 12, 2011
  4. Merge "HealthManager varz changes - Fixed varz stats keeping framewor…

    …ks/runtimes around that are no longer in the DB - Added started apps and instances - Aggregated queries to reduce number of requests"
    Vadim Spivak committed with Gerrit Code Review Aug 12, 2011
  5. HealthManager varz changes

    - Fixed varz stats keeping frameworks/runtimes around that are no longer in the DB
    - Added started apps and instances
    - Aggregated queries to reduce number of requests
    
    Change-Id: I03ee8368e1e2f06819f40cb1cc74ced949f8a79f
    Vadim Spivak committed Aug 12, 2011
  6. bump services submodule ptr

    Change-Id: Ibfd057943906f5b2ecd6bf875478d25ad9742244
    Patrick Bozeman committed Aug 12, 2011
  7. bump test submodule ptr

    Change-Id: I104e4c7872d0b2d353e7f6dc1b9072afe86c8c33
    Patrick Bozeman committed Aug 12, 2011
  8. bump java submodule ptr

    Change-Id: Ia879da24f35bdd70afcb90b8e6b1093ef670f6d7
    Patrick Bozeman committed Aug 12, 2011
  9. be very explict about rvm ruby versions

    Change-Id: Ib5e6270d22dee67e7886cc0f5936c5dd1761a0b4
    Patrick Bozeman committed Aug 11, 2011
  10. fix File.expand_path usage.

    For applying this fix, you can use MySQL (/w em_mysql2 adapter) as cloud controller database by configuring cloud_controller.yml as follows:
    
      development:
        database: cloudcontroller
        host: localhost
        port: 3306
        username: root
        password: password
        adapter: em_mysql2
        encoding: utf8
        timeout: 2000
    
    Change-Id: Ie881f617f77d1d4aacc0772b1594547b9beb1af5
    yssk22 committed with Patrick Bozeman Aug 5, 2011
  11. Fix typo when checking for preexisting Erlang installation in vcap_setup

    Change-Id: I59e37fd76a18d6fcc742aded67d37a4dda5117b9
    oldpatricka committed with Patrick Bozeman Aug 9, 2011
Commits on Aug 10, 2011
  1. Include the PG drivers in the tomcat bits.

    Change-Id: I2ef89a26a10715c67e7018d197f4a11c61714fc7
    AB Srinivasan committed Aug 10, 2011
  2. Merge "Simple change to fix deployments that should not download vcap…

    … repo"
    mahpat committed with Gerrit Code Review Aug 10, 2011
  3. Support for Plain Java and refactored staging.

    Change-Id: I88ad35dc9a8783ade4f18cede267ac4e237d607b
    AB Srinivasan committed Aug 10, 2011
Commits on Aug 9, 2011
  1. Simple change to fix deployments that should not download vcap repo

    Setting a default value for REVISION was incorrect. nil is the right default for
    this property.
    
    Change-Id: Iaf2d636cb508c4e3469f479663f9ade8ecf9f895
    mahpat committed Aug 8, 2011
  2. Bump up Tomcat version used in Cloud Foundry to 6.0.32

    Change-Id: Id48333d0a694d7dd33f0e3d47a1b61e399371285
    AB Srinivasan committed Aug 9, 2011
Commits on Aug 4, 2011
  1. Merge branch 'fixacls'

    Change-Id: Ic8056c260937a317f70cc24c125abf8c80aa2d06
    kushmerick committed Aug 4, 2011
  2. Allow service offerings to switch from private to public

    Change-Id: I5d1ba7483949b8de459fa8e48e3cf065de5c982c
    kushmerick committed Aug 4, 2011
  3. Merge "Multi host setup scripts"

    mahpat committed with Gerrit Code Review Aug 4, 2011
Commits on Aug 3, 2011
  1. Merge "Initial pass at stand alone stager component"

    mpage committed with Gerrit Code Review Aug 3, 2011
  2. Merge "Merge branch 'services-r3'"

    kushmerick committed with Gerrit Code Review Aug 3, 2011
  3. Multi host setup scripts

    All deployments (multi host and single host) are driven through a templated
    config file. Look at dev_setup/deployments/sample* for an example of what this
    config file looks like. There is sufficient documentation inside the configs.
    But at a high level the user specifes "jobs" to "install" and jobs that are
    "installed". The "installed" jobs have properties associated with them which are
    used by jobs that are in the "install" list. e.g. nats could be an installed job
    and its properties like host/port will be used by the jobs in the install list.
      The deployment code now goes through this config file, does sanity checking to
    verify valid job names, valid properties etc. It leverages "rake" to manage
    dependencies between jobs, look at dev_setup/lib/job_*. So we enforce that all
    dependent jobs for a given job are either in the "installed" list or in the
    "install" list.  Once all job specs are verified, we generate the chef runlist
    on the fly and the deployment proceeds to install the required components using
    chef.
    
    NOTE:
    1. For now, multi host service deployment is limited to only selecting a service
    say "redis" and the scripts will install the service node, gateway and the redis
    software itself on the same host. We can add more flexibility to this later.
    2. For now, we use roles to enforce chef recipe dependencies. It seems to be
    working well for what we have right now and I like the fact that there is one
    location that maintains the chef dependencies for a given component.
    3. For now, not all configurations of multi host are tested. In fact, I have
    only
    verified NATS. The config file template changes for all the other components
    will be added in later changes. e.g. with this change you cannot run ccdb on a
    separate host and expect cloud controller to work. But the changes to make it
    work are more about adding the right templated fields and testing those changes.
    
    Testing Done: I have verified that I can install "nats" on one box and the rest
    of the components on a different box and things work fine.
    
    Change-Id: I165b01fd65e4283748cf2cf9b2438369ae6332ce
    mahpat committed Jul 30, 2011