Cloud Foundry Cloud Controller
Switch branches/tags
Nothing to show
Clone or download
tcdowney Merge pull request #1236 from cf-routing/batch_copilot_sync
add batching to Copilot::Sync

[#161301937]
Latest commit adc4dd5 Oct 19, 2018
Permalink
Failed to load latest commit information.
app Merge pull request #1223 from cloudfoundry-incubator/pr-wip-service-u… Oct 19, 2018
bin Upgrade Rails to 5.2.0 Oct 3, 2018
config Fix remaining failing tests Oct 3, 2018
db Merge pull request #1223 from cloudfoundry-incubator/pr-wip-service-u… Oct 19, 2018
docs Merge pull request #1239 from cloudfoundry-incubator/pr-docs-update Oct 19, 2018
lib Merge pull request #1236 from cf-routing/batch_copilot_sync Oct 19, 2018
middleware Bump rubocop from 0.51.0 to 0.52.1 Jul 27, 2018
scripts Move which-capi.sh script to which_capi in capi-workspace repo Oct 5, 2018
spec Merge pull request #1236 from cf-routing/batch_copilot_sync Oct 19, 2018
vendor/errors Merge pull request #1224 from cloudfoundry-incubator/pr-improved-erro… Oct 2, 2018
.codeclimate.yml Reek is too opinionated. Aug 4, 2017
.csslintrc Audit what files codeclimate is inspecting Aug 3, 2017
.eslintignore Audit what files codeclimate is inspecting Aug 3, 2017
.eslintrc.yml Audit what files codeclimate is inspecting Aug 3, 2017
.gitignore Revert "Revert "Implement DB encryption key rotation."" Dec 13, 2017
.rspec Make rspec backtraces smaller Apr 20, 2016
.rspec_parallel Revert "Revert "Use parallel_tests to run tests in parallel"" Oct 7, 2016
.rubocop.yml Bump rubocop from 0.52.0 to 0.58.2 Aug 17, 2018
.ruby-version Upgrade to ruby 2.4 Sep 23, 2017
.snyk Update nokogiri to 1.8.5 Oct 15, 2018
.travis.yml Use Code Climate's new test reporter for test coverage Sep 27, 2017
CONTRIBUTING.md update link to `create_and_upload` script Aug 31, 2018
Gemfile Bump rubyzip back to master, 1.2.2 Oct 4, 2018
Gemfile.lock Update nokogiri to 1.8.5 Oct 15, 2018
Guardfile Bump Rubocop to 0.48.1 May 27, 2017
ISSUE_TEMPLATE.md Fixing typo in current result for issue template Oct 17, 2018
LICENSE Revert "Update License and Notice for project" May 23, 2016
NOTICE Streamline multi-copyright notice Feb 3, 2017
PULL_REQUEST_TEMPLATE.md Update PULL_REQUEST_TEMPLATE.md Oct 17, 2017
README.md Added experimental in README for Alicloud Jun 21, 2018
Rakefile Bundle install before running check_docs_links Oct 16, 2018
manifest.yml Server-side manifests: simplify error-message generation Mar 13, 2018

README.md

Build Status Code Climate Code Climate slack.cloudfoundry.org

Welcome to the Cloud Controller

Helpful Resources

Components

Cloud Controller

The Cloud Controller provides REST API endpoints to create and manage apps, services, user roles, and more!

Database

The Cloud Controller supports Postgres and Mysql.

Blobstore

The Cloud Controller manages a blobstore for:

  • Resource cache: During package upload resource matching, Cloud Controller will only upload files it doesn't already have in this cache.
  • App packages: Unstaged files for an application
  • Droplets: An executable containing an app and its runtime dependencies
  • Buildpacks: Set of programs that transform packages into droplets
  • Buildpack cache: Cached dependencies and build artifacts to speed up future staging

Cloud Controller currently supports webdav and the following fog connectors:

  • Alibaba Cloud (Experimental)
  • Azure
  • Openstack
  • Local (NFS)
  • Google
  • AWS

Runtime

The Cloud Controller uses Diego to stage and run apps and tasks.

See Diego Design Notes for more details.

Contributing

Please read the contributors' guide and the Cloud Foundry Code of Conduct

Unit Tests

TLDR: Always run bundle exec rake before committing

To maintain a consistent and effective approach to testing, please refer to the spec README and keep it up to date, documenting the purpose of the various types of tests.

By default rspec will randomly pick between postgres and mysql.

It will try to connect to those databases with the following connection string:

  • postgres: postgres://postgres@localhost:5432/cc_test
  • mysql: mysql2://root:password@localhost:3306/cc_test

To specify a custom username, password, host, or port for either database type, you can override the default connection string prefix (the part before the cc_test database name) by setting the MYSQL_CONNECTION_PREFIX and/or POSTGRES_CONNECTION_PREFIX variables. Alternatively, to override the full connection string, including the database name, you can set the DB_CONNECTION_STRING environment variable. This will restrict you to only running tests in serial, however.

For example, to run unit tests in parallel with a custom mysql username and password, you could execute:

MYSQL_CONNECTION_PREFIX=mysql2://custom_user:custom_password@localhost:3306 bundle exec rake

The following are examples of completely fully overriding the database connection string:

DB_CONNECTION_STRING="postgres://postgres@localhost:5432/cc_test" DB=postgres rake spec:serial
DB_CONNECTION_STRING="mysql2://root:password@localhost:3306/cc_test" DB=mysql rake spec:serial

If you are running the integration specs (which are included in the full rake), and you are specifying DB_CONNECTION_STRING, you will also need to have a second test database with _integration_cc as the name suffix.

For example, if you are using:

DB_CONNECTION_STRING="postgres://postgres@localhost:5432/cc_test"

You will also need a database called:

`cc_test_integration_cc`

The command

rake db:create

will create the above database when the DB environment variable is set to postgres or mysql. You should run this before running rake in order to ensure that the cc_test database exists.

Running tests on a single file

The development team typically will run the specs to a single file as (e.g.)

bundle exec rspec spec/controllers/runtime/users_controller_spec.rb

Running all the unit tests

bundle exec rake spec

Note that this will run all tests in parallel by default. If you are setting a custom DB_CONNECTION_STRING, you will need to run the tests in serial instead:

bundle exec rake spec:serial

To be able to run the unit tests in parallel and still use custom connection strings, use the MYSQL_CONNECTION_PREFIX and POSTGRES_CONNECTION_PREFIX environment variables described above.

Running static analysis

bundle exec rubocop

Running both unit tests and rubocop

By default, bundle exec rake will run the unit tests first, and then rubocop if they pass. To run rubocop first, run:

RUBOCOP_FIRST=1 bundle exec rake

CF Acceptance Tests (CATs)

To ensure our changes to the Cloud Controller correctly integrate with the rest of the Cloud Foundry components like Diego, we run the CF Acceptance Tests (CATs) against a running CF deployment. This test suite uses the CF CLI to ensure end-user actions like cf push function end-to-end.

For more substantial code changes and PRs, please deploy your changes and ensure that at least the core CATs suite passes. Follow the instructions here for setting up the CATs suite. The following will run the core test suites against a local bosh-lite:

cd ~/go/src/github.com/cloudfoundry/cf-acceptance-tests
cat > integration_config.json <<EOF
{
  "api": "api.bosh-lite.com",
  "apps_domain": "bosh-lite.com",
  "admin_user": "admin",
  "admin_password": "admin",
  "skip_ssl_validation": true
}
EOF
export CONFIG=$PWD/integration_config.json
./bin/test -nodes=3

If your change touches a more specialized part of the code such as Isolation Segments or Tasks, please opt into the corresponding test suites. The full list of optional test suites can be found here.

Logs

Cloud Controller uses Steno to manage its logs. Each log entry includes a "source" field to designate which module in the code the entry originates from. Some of the possible sources are 'cc.app', 'cc.app_stager', and 'cc.healthmanager.client'.

Here are some use cases for the different log levels:

  • error - the CC received a malformed HTTP request, or a request for a non-existent droplet
  • warn - the CC failed to delete a droplet, CC received a request with an invalid auth token
  • info - CC received a token from UAA, CC received a NATS request
  • debug2 - CC created a service, updated a service
  • debug - CC syncs resource pool, CC uploaded a file

Configuration

The Cloud Controller uses a YAML configuration file. For an example, see config/cloud_controller.yml.