Release and Stemcell Updates

Release Old Version New Version
backup-and-restore-sdk 1.13.0 1.14.0
mapfs 1.1.3 1.1.4
Assets 2

@heyjcollins heyjcollins released this Feb 11, 2019 · 5 commits to master since this release

Updated Ops-files

  • operations/experimental/enable-bpm-garden.yml & operations/experimental/rootless-containers.yml
    • enable-bpm-garden.yml is now a no-op. It's contents have been integrated into rootless-containers.yml so a single file will handle all required configuration to:
      1. turn on rootless properties in Garden
      2. disable containerd mode
      3. enable BPM for the Garden job

(This is the preexisting rootless configuration, we are just moving properties across yml files)

Release and Stemcell Updates

Release Old Version New Version
cf-syslog-drain 8.2 9.0
cf-smoke-tests 40.0.44 40.0.45
garden-runc 1.18.1 1.18.2
statsd-injector 1.6.0 1.7.0
Assets 2

@heyjcollins heyjcollins released this Feb 7, 2019 · 13 commits to master since this release

Updated Ops-files

  • uaas-support/openstack/cloud-config.yml
    • Reduce OpenStack loadbalancer_pools to 10 instead of 100 (to be consistent with the cf-d v7.0 manifest change to use reduce the reservable ports from 100 to 10 #660).

Release and Stemcell Updates

Release Old Version New Version
binary-buildpack 1.0.28 1.0.30
bits-service 2.22.0 2.23.0
capi 1.74.0 1.76.0
java-buildpack 4.17.1 4.17.2
go-buildpack 1.8.31 1.8.33
garden-runc 1.18.0 1.18.1
dotnet-core-buildpack 2.2.4 2.2.5
diego 2.27.0 2.28.0
nginx-buildpack 1.0.4 1.0.8
php-buildpack 4.3.68 4.3.70
nodejs-buildpack 1.6.40 1.6.43
python-buildpack 1.6.27 1.6.28
ruby-buildpack 1.7.29 1.7.31
staticfile-buildpack 1.4.37 1.4.39
log-cache 2.1.0 2.1.1
loggregator-agent 3.3 3.5
mapfs 1.1.2 1.1.3
nfs-volume 1.7.6 1.7.7
windows2016fs 1.13.0 2.0.0
postgres 34 35
windows1803fs 1.9.0 2.0.0
winc 1.7.0 1.8.0
ubuntu-xenial 170.24 250.4
Assets 2

@heyjcollins heyjcollins released this Jan 31, 2019 · 74 commits to master since this release

Manifest Updates

  • Default Cloud Controller to retrieve container metrics directly from log-cache deb25ef
    • Cloud Controller supports both retrieving container metrics directly
      from log-cache and from trafficcontroller. However, trafficcontroller
      itself is backed by log-cache and this change will enable Cloud
      Controller to consume the upcoming metrics tags functionality.


Deprecated Ops-files

  • operations/experimental/use-logcache-for-cloud-controller-app-stats.yml - inlined to base manifest.

Release and Stemcell Updates

Release Old Version New Version
bpm 1.0.0 1.0.2
uaa 68.0 69.0
backup-and-restore-sdk 1.12.0 1.13.0
ubuntu-xenial 170.21 170.24
windows2012R2 1200.27 1200.29
Assets 2

@heyjcollins heyjcollins released this Jan 24, 2019 · 86 commits to master since this release


NGINX-Buildpack and R-Buildpack releases for cflinuxfs3 are now available.

Manifest Updates

  • cflinuxfs3 R and NGINX buildpacks added
  • Removed unneeded containerd job

Updated Ops-files

  • operations/experimental/use-native-garden-runc-runner.yml
  • Removed instruction to remove the containerd job since it's removed from the base manifest.
  • operations/backup-and-restore/skip-backup-restore-droplets-and-packages.yml & operations/backup-and-restore/skip-backup-restore-droplets.yml
    • Add support so selective backups can be executed against foundations using internal blobstores.

Release and Stemcell Updates

Release Old Version New Version
cflinuxfs3 0.49.0 0.51.0
cflinuxfs2 1.259.0 1.260.0
java-buildpack 4.16.1 4.17.1
go-buildpack 1.8.30 1.8.31
garden-runc 1.17.2 1.18.0
dotnet-core-buildpack 2.2.3 2.2.4
diego 2.25.0 2.27.0
php-buildpack 4.3.67 4.3.68
nodejs-buildpack 1.6.38 1.6.40
nginx-buildpack X 1.0.4
r-buildpack X 1.0.3
python-buildpack 1.6.25 1.6.27
log-cache 2.0.2 2.1.0
haproxy 9.3.0 9.4.1
windows2016fs 1.12.0 1.13.0
postgres 33 34
hwc-buildpack 3.1.3 3.1.4
windows1803fs 1.8.0 1.9.0
smb-volume 0.2.6 1.0.0
bits-service 2.21.0 2.22.0
ubuntu-xenial 170.19 170.21
windows1803 1803.5 1803.6
windows2016 1709.16 1709.17
Assets 2

@heyjcollins heyjcollins released this Jan 17, 2019 · 140 commits to master since this release

💥Breaking change release may require multiple deployments💥

〰️ Breaking Changes 〰️

  • PXC replaces cf-mysql as the default DB
    • If your foundation is backed by cf-mysql, you must follow the Migrate to PXC steps provided at the bottom of these release notes.
    • You can skip the Migrate to PXC steps and deploy cf-deployment v7.0.0 directly if any of the following applies to your foundation:
      1. You're deploying a fresh foundation
      2. You're upgrading an existing foundation that uses (or migrating an existing foundation to start using)
        • an external database (your bosh deploy includes operations/use-external-dbs.yml)
        • Postgres (your bosh deploy includes operations/use-postgres.yml)

Other Notable Changes

  • cflinuxfs3 replaces cflinuxfs2 as the default stack
    • cflinuxfs2 will be remain available to support dev migration of existing apps to cflinuxfs3.
    • In March 2019, cflinuxfs2 will be removed from cf-deployment and apps that haven't been migrated by that time will crash.
  • Use 10 reservable ports instead of 100 -- #660
    • Manual steps are required to update the reservable ports via API.
      See Update Reservable Ports instructions at the bottom of release notes.

Manifest Updates

  • operations/use-pxc.yml has been inlined
  • The cc.default_stack property has been added and set to cflinuxfs3
  • The instance-groups/api/routing-api/router-groups/default-tct/reservable-ports property has been updated from 1024-1123 to 1024-1033
  • uaa/scim/users/name: bosh has been removed because it is no longer being used by cf-smoke-tests.


Updated Ops-files

  • operations/use-postgres.yml &
    operations/use-external-dbs.yml &
    operations/scale-database-cluster.yml &
    operations/experimental/enable-tls-cloud-controller-postgres.yml &
    operations/enable-nfs-volume-service.yml &
    • All the ops files above have been updated to to reference correct jobs/properties now that PXC has replaced cf-mysql in cf-deployment.yml.
  • operations/bosh-lite.yml
    • Add in uaa ca cert to make sure cf ssh continues to work.
      with ssl validation
  • iaas-support/bosh-lite/cloud-config.yml
    • Remove the credhub-lb cloud-properties because we've deprecated add-credhub-lb.yml.

Deleted Ops-files

  • operations/backup-and-restore/enable-backup-restore-smb-broker.yml
  • operations/experimental/add-cflinuxfs3.yml
  • operations/experimental/enable-routing-integrity.yml
  • operations/experimental/use-latest-windows1803-stemcell.yml
  • operations/experimental/use-pxc-for-smb-volume-service.yml
  • operations/experimental/windows1803-cell.yml
  • operations/experimental/use-garden-containerd.yml

Deprecated Ops-files

  • operations/use-pxc.yml
  • operations/use-pxc-for-nfs-volume-service.yml
  • operations/migrate-cf-mysql-to-pxc.yml
  • operations/experimental/add-credhub-lb.yml
  • operations/experimental/perm-service.yml
  • operations/experimental/perm-service-with-pxc-release.yml
  • operations/experimental/perm-service-with-tcp-routing.yml
  • operations/experimental/enable-mysql-tls.yml

Dropped symlinks

  • operations/experimental/enable-routing-integrity.yml -> ../enable-routing-integrity.yml
  • operations/experimental/use-latest-windows1803-stemcell.yml -> ../use-latest-windows1803-stemcell.yml
  • operations/experimental/windows1803-cell.yml -> ../windows1803-cell.yml

Release and Stemcell Updates

Release Old Version New Version
loggregator 104.4 104.5
loggregator-agent 3.2 3.3
statsd-injector 1.5.0 1.6.0
cf-syslog-drain 8.1 8.2
cflinuxfs2 1.257.0 1.259.0
cflinuxfs3 0.48.0 0.49.0
cf-cli 1.11.0 1.12.0
bits-service 2.20.0 2.21.0
cf-mysql 36.17.0 X
perm 0.0.7 X
ubuntu-xenial 170.16 170.19

Steps Required to migrate from cf-mysql to pxc-mysql

If your existing foundation is backed by cf-mysql, two deployments are required to migrate from cf-mysql to pxc-mysql, both of which will incur downtime.

  1. Your first migration deployment will be run using cf-d release v6.x.0 and include the operations/migrate-cf-mysql-to-pxc.yml ops file.
  2. Your second deployment will upgrade to cf-d release v7.0.0.

Deployment downtime details:
The amount of downtime to expect during the migration deployment will depend on the size of your database, the VM configuration, and other factors.
There will also be a short period of downtime during the second deployment while BOSH stops the single VM with the database instance.

To estimate the length of downtime you can expect during the first migration deployment you can perform a manual backup without saving the backup file and note how long that procedure takes to complete
To perform a manual backup, run the following command:

time /var/vcap/packages/mariadb/bin/mysqldump -u root -p --all-databases > /dev/null

BOSH runs additional processes before and after a MySQL migration, but the total time the migration takes is not much longer than a manual backup.

The impact of the MySQL downtime

  • While the MySQL database is not available users will not be able to push apps, scale apps, or run other cf CLI commands.
  • UAA will also not be available, so any other services backed by UAA will also be impacted.
  • There will be no app downtime save for apps which use TCP routing. TCP routes will not be routable during the migration.

The first migration deployment will:

  • scale your database instance group to a single node
  • make a backup copy of your migrated MySQL data on the database instance
  • add the pxc job to the database instance group (this job is added in-addition-to the cf-mysql job that's already running there)
  • start using the pxc-release instead of the cf-mysql release

Before you deploy...
Make sure there's sufficient disk on your database instance to store the backup copy that will be generated.

  • ssh to the database instance: bosh ssh database/0
  • View disk size/usage: df -h
  • Check the Use% for the disk mounted on /var/vcap/store
    If your Use% is >= 40%, you need to increase disk capacity.

You can increase capacity by creating a custom ops file that increases the persistent_disk_type value for your database instance and including that ops file in your migration deployment.
For example, to increase the size of your persistent disk beyond the default 10GB, your ops file would look like this:

- type: replace
  path: /instance_groups/name=database/persistent_disk_type
  value: 50GB

Assuming you've completed the preparations described above, you can safely run the migration deployment:

  1. Check out cf-deployment release v6.10.0
    ❗️Make sure you're NOT deploying cf-deployment release v7.0.0 for the initial migration deploy.
    ❗️Make sure you don't include operations/scale-database-cluster.yml in the migration deployment (The migrate-cf-mysql-to-pxc.yml ops file scales the database to a single node).
  2. Run your bosh deploy command.
    It will look something like this:
    bosh deploy -d cf cf-deployment.yml -v -o operations/migrate-cf-mysql-to-pxc.yml
  3. Execute the post-deployment validations you typically run.

Assuming your migration deployment was successful, you're ready to deploy cf-d v7.0.0.

The second v7.0.0 deployment will:

  • remove the cf-mysql job from the database instance group
  • you may optionally scale your database back up at this point
  • it will not delete the database backup that was created in the migration deployment (you must do this manually).

Before you deploy, you may want to take action regarding the migrated mysql backup to free up disk space and, if applicable, make it possible to scale your persistent disk back down to the size it was before you executed the migration deployment.

  • *Note - PXC requires a minimum of 5GB persistent disk. The default disk set in the base manifest is 10GB, so this shouldn't be a problem, but this information is provided here in the off-chance there are operators running custom configurations with smaller disk allocations for the database instance.

Run the following command to ssh to the database instance group and view the contents stored on persistent disk:

bosh ssh database/0
sudo su
cd /var/vcap/store

From /var/vcap/store/ you'll see a directory called mysql-migration-backup.
Assuming you've taken appropriate steps to confirm your foundation features/functions are stable, you may decide to make a copy of this backup or delete the mysql-migration-backup directory and its contents.

If you decide to keep the backup for the time being and you had to increase the persistent_disk as part of the migration deployment, your deploy of cf-d v7.0.0 will also need to include the custom ops file you created to increase disk capacity in the migration deployment.

Assuming you've reasoned about and completed any actions related to the comments above, you can run the cf-d v7.0.0 deployment:

  1. check out cf-deployment release v7.0.0
  2. run your bosh deploy command.
    It will look something like this:
    bosh deploy -d cf cf-deployment.yml -v
  3. Execute the post-deployment validations you typically run.

Update Reservable Ports Instructions

  1. Retrieve router group GUID for the default-tcp router group:
    cf curl /routing/v1/router_groups
  2. Update reservable ports range.
    ❗️Warning❗️ If routes are registered for ports that are not in the new range, modifying your load balancer to remove these ports will result in backends for those routes becoming inaccessible.
    cf curl -X PUT /routing/v1/router_groups/<ROUTER_GROUP_GUID> -d '{"reservable_ports":"1024-1033"}'
Assets 2

@heyjcollins heyjcollins released this Jan 16, 2019 · 172 commits to master since this release

Updated Ops-files

  • operations/experimental/enable-routing-integrity-windows1803.yml & operations/experimental/enable-routing-integrity-windows2016.yml
  • Pull envoy-release from instead of GitHub.

Release and Stemcell Updates

Release Old Version New Version
binary-buildpack 1.0.27 1.0.28
cf-mysql 36.16.0 36.17.0
cf-smoke-tests 40.0.43 40.0.44
cflinuxfs3 0.47.0 0.48.0
cflinuxfs2 1.256.0 1.257.0
go-buildpack 1.8.29 1.8.30
garden-runc 1.17.1 1.17.2
dotnet-core-buildpack 2.2.0 2.2.3
python-buildpack 1.6.23 1.6.25
php-buildpack 4.3.64 4.3.67
nodejs-buildpack 1.6.34 1.6.38
ruby-buildpack 1.7.27 1.7.29
staticfile-buildpack 1.4.35 1.4.37
nfs-volume 1.7.5 1.7.6
hwc-buildpack 3.1.2 3.1.3
envoy 0.1.0 0.3.0
windows-syslog 1.0.1 1.0.3
bits-service 2.19.0 2.20.0
ubuntu-xenial 170.14 170.16
windows2012R2 1200.26 1200.27
Assets 2

@heyjcollins heyjcollins released this Jan 2, 2019 · 197 commits to master since this release

Updated Ops-files

  • operations/experimental/enable-oci-phase-1.yml
    • Enable declarative download on Cloud Controller components.

Release and Stemcell Updates

Release Old Version New Version
bits-service 2.18.0 2.19.0
cflinuxfs3 0.46.0 0.47.0
cflinuxfs2 1.255.0 1.256.0
uaa 66.0 68.0
postgres 31 33
windows2016fs 1.11.0 1.12.0
windows1803fs 1.7.0 1.7.0
windows2016 1709.15 1709.16
windows1803 1803.4 1803.5
ubuntu-xenial 170.13 170.14
Assets 2

@heyjcollins heyjcollins released this Dec 21, 2018 · 212 commits to master since this release

Updated Ops-files

  • operations/enable-nfs-ldap.yml & operations/enable-nfs-volume-service.yml & operations/experimental/enable-nfs-volume-service-credhub.yml & operations/experimental/enable-smb-volume-service.yml
    • Communication between the rep and the NFS and SMB drivers is now secured through mutual TLS.

Release and Stemcell Updates

Release Old Version New Version
diego 2.24.0 2.25.0
windows2016fs 1.11.0 1.12.0
Assets 2

@heyjcollins heyjcollins released this Dec 18, 2018 · 225 commits to master since this release

Manifest Updates

  • Update cf-smoke-tests release to run with client rather than user credentials
    • Add uaa/scim/clients/cf_smoke_tests client and update cf-smoke-tests to use it rather than uaa/scim/users/bosh.
  • Move windows1803 bosh-dns-alias to the base manifest
    • For windows cells in the past we've added dns aliases to the base manifest so that deployments enabling windows cells via windows*.yml ops files aren't required to update all VMs in the foundation.
  • Switch deprecated property experimental_containerd_mode to containerd_mode since containerd is no longer experimental and is enabled by default. #680


New Ops-files

  • operations/backup-and-restore/skip-backup-restore-droplets.yml &operations/backup-and-restore/skip-backup-restore-droplets-and-packages.yml
    • Operators can now skip the blobstore droplets and/or packages buckets when taking a backup using bbr. #682

Updated Ops-files

  • operations/windows1803-cell.yml & operations/windows2016-cell.yml
    • Install the 'windows' stack associated binary-buildpack and hwc-buildpack #675
  • operations/experimental/enable-bpm-garden.yml & operations/experimental/rootless-containers.yml & operations/experimental/use-native-garden-runc-runner.yml
    • Change deprecated property experimental_containerd_mode to containerd_mode. #680
  • operations/experimental/enable-oci-phase-1.yml
    • Add rep two-layer layering mode to oci phase 1 ops file. Diego story

Release and Stemcell Updates

Release Old Version New Version
cf-smoke-tests 40.0.42 40.0.43
cf-networking 2.19.0 2.20.0
cflinuxfs2 1.254.0 1.255.0
silk 2.19.0 2.20.0
uaa 67.0 66.0
nfs-volume 1.7.4 1.7.5
hwc-buildpack 3.1.1 3.1.2
smb-volume 0.2.5 0.2.6
backup-and-restore-sdk 1.11.1 1.11.2
windows2012R2 1200.25 1200.26
ubuntu-xenial 170.12 170.13
Assets 2