Diego v2.0.0

Changes from v1.35.0 to v2.0.0

• Verified with cf-deployment @ 3301bea.
• Verified with garden-runc-release v1.11.1.
• Verified with garden-windows-bosh-release v0.13.0.
• Verified with cflinuxfs2-release v1.188.0.

IMPORTANT: Diego v2.0.0 introduces some breaking changes to the operator configuration of the BOSH release and deliberately removes support for other job properties and API endpoints that will be removed soon. In particular, in this release:

• Diego APIs now require configuration for mutual TLS.
• Diego components default to using Locket as their lock and cell-registration provider.
• The BBS API no longer supports migration of legacy etcd data to MySQL or Postgres.
• The legacy spiff-based manifest-generation system is removed in favor of cf-deployment.

A full list of expected changes in the v2 release series is available in the v2.0.0 proposal.

Significant changes

BBS API

• Consider fixing interfaces to be consistent between when we specify actual lrp key vs index and process guid as separate parameters
• As a Diego contributor, I expect the BBS documentation to explain the BBS migration system and the constraints on individual migrations so that I can write safe migrations

BBS Benchmarks

• Create an ops-file to run the BBS benchmarks on bosh-lite on a cf-deployment Diego

BBS Relational Datastore

• As a Diego operator, I expect to be able to remove all etcd-related configuration from my BBS and benchmark-BBS properties so that I can simplify my deployment manifests

cfdot

• As a cfdot user, I expect the flags and corresponding environment variables deprecated before Diego v2 to be removed so that the cfdot configuration is simpler

De-Consuling Locks

• As a Diego operator, I expect the BBS, the auctioneer, the Diego cell reps, and the route-emitter in "global" mode all to default to the use of Locket for their component locks and registrations so that I can migrate away from Consul by default

Per-Instance Proxy (Experimental)

• Route integrity should allow routing to app when app process binds to only the internal container IP

Manifest Generation

• As a Diego operator, I expect diego-release no longer to supply manifest-generation scripts and templates so that I am not confused when they no longer work with the unsupported cf-release manifest-generation system

Component Logging and Metrics

• Clean up error handling in BBS sqldb driver

Dependencies

• As a Diego operator, I expect diego-release to use Golang 1.9.4 so that I am patched against security vulnerabilities
• As a Diego operator, I expect the diego-release documentation to inform me of the updated minimum garden, BOSH, and SQL dependencies for v2 so that I can ensure I have updated them to be compatible for this major release version

Test Suites and Tooling

• Stop running "classic" dusts after final diego-release v1.x exists
• Bump vizzini eventually timeout to 15 seconds only for tests that use privileged containers
• Increase the SQLDB shutdown timeout
• Fix flaky route-emitter test
• cloudfoundry/bbs #26: Remove the db/etcd test suite from the bbs

Security

• As a Diego operator, I expect the BBS, auctioneer, and rep components all to require mutual TLS to secure their API servers so that I can ensure their security against remote clients

BOSH job changes

None.

BOSH property changes

auctioneer

• Added locks.locket.enabled, defaulting to true.

• Changed diego.auctioneer.locket.api_location default to locket.service.cf.internal:8891.

• Removed diego.auctioneer.bbs.require_ssl.

• diego.auctioneer.bbs.ca_cert is now required.

• diego.auctioneer.bbs.client_cert is now required.

• diego.auctioneer.bbs.client_key is now required.

• diego.auctioneer.ca_cert is now required.

• diego.auctioneer.server_cert is now required.

• diego.auctioneer.server_key is now required.

bbs

• Added cell_registrations.locket.enabled, defaulting to true.

• Added locks.locket.enabled, defaulting to true.

• Changed diego.bbs.locket.api_location default to locket.service.cf.internal:8891.

• Removed diego.bbs.desired_lrp_creation_timeout.

• Removed diego.bbs.etcd.ca_cert.

• Removed diego.bbs.etcd.client_cert.

• Removed diego.bbs.etcd.client_key.

• Removed diego.bbs.etcd.client_session_cache_size.

• Removed diego.bbs.etcd.machines.

• Removed diego.bbs.etcd.max_idle_conns_per_host.

• Removed diego.bbs.etcd.require_ssl.

• Removed diego.bbs.require_ssl.

• diego.bbs.ca_cert is now required.

• diego.bbs.server_cert is now required.

• diego.bbs.server_key is now required.

benchmark-bbs

• Removed benchmark-bbs.bbs.require_ssl.

• Removed benchmark-bbs.etcd.ca_cert.

• Removed benchmark-bbs.etcd.client_cert.

• Removed benchmark-bbs.etcd.client_key.

• Removed benchmark-bbs.etcd.client_session_cache_size.

• Removed benchmark-bbs.etcd.machines.

• Removed benchmark-bbs.etcd.max_idle_conns_per_host.

• Removed benchmark-bbs.etcd.require_ssl.

• benchmark-bbs.bbs.ca_cert is now required.

• benchmark-bbs.bbs.client_cert is now required.

• benchmark-bbs.bbs.client_key is now required.

cfdot

• Removed diego.cfdot.bbs.use_ssl.

• tls.ca_certificate is now required.

• tls.certificate is now required.

• tls.private_key is now required.

rep and rep_windows

• Added cell_registrations.locket.enabled, defaulting to true.

• Changed diego.rep.locket.api_location default to locket.service.cf.internal:8891.

• Removed admin_api.require_tls.

• Removed diego.rep.bbs.ca_cert.

• Removed diego.rep.bbs.client_cert.

• Removed diego.rep.bbs.client_key.

• Removed diego.rep.ca_cert.

• Removed diego.rep.require_tls.

• Removed diego.rep.server_cert.

• Removed diego.rep.server_key.

• Removed use_v2_tls.

• tls.ca_cert is now required.

• tls.cert is now required.

• tls.key is now required.

route_emitter and route_emitter_windows

• Changed locks.locket.enabled default to true.

• Removed diego.route_emitter.bbs.require_ssl.

• diego.route_emitter.bbs.ca_cert is now required.

• diego.route_emitter.bbs.client_cert is now required.

• diego.route_emitter.bbs.client_key is now required.

ssh_proxy

• Removed diego.ssh_proxy.bbs.require_ssl.

• diego.ssh_proxy.bbs.ca_cert is now required.

• diego.ssh_proxy.bbs.client_cert is now required.

• diego.ssh_proxy.bbs.client_key is now required.

vizzini

• Removed vizzini.bbs.require_ssl.

• vizzini.bbs.client_cert is now required.

• vizzini.bbs.client_key is now required.

BOSH link changes

None.