Releases: gstackio/gstack-bosh-environment
GBE & Easy Foundry v4.1.1
Improvements
- Traefik now runs smoke tests after being deployed.
- Bumped the
broker-registrar
release to v3.5.1, removing the workaround implemented incassandra
,data-services
,postgres
andrabbitmq
subsystems. - Logsearch memory consumtion has been reduced.
- Version updates, see below.
Fixes
-
Fixed the startup timeout of Kibana in the
logsearch
subsystem. -
The Postgres subsystem was not working because Prometheus v23.3.0 is not compatible with the Postgres v11.x shipped by the Postgres release v36. This version of Easy Foundry properly pins the Postgres version to
v31
for Prometheus.A next release version of Easy Foundry will upgrade both the Postgres database and the Prometheus version. The tested upgrade path is the following:
- Prometheus
v23.3.0
+ Postgresv31
- Prometheus
v25.0.0
+ Postgresv32
(and follow instruction from these release notes) - Prometheus
v25.0.0
+ Postgresv36
- Prometheus
Notice
log-cache
is disabled in Cloud Foundry, because of excessive memory consumption in BOSH-Lite.
Components Versions
Component | New Version | Old Version |
---|---|---|
Træfik | 1.6.0 | 1.5.0 |
GBE & Easy Foundry v4.0.0
💥 Kown issues notice 💥
- ❗️💥 Do not converge the
prometheus
subsystem with the Postgres release v36 shipped in this v4.0.0 of Easy Foundry. Instead, pin the Postgres release version to v31 like this inprometheus/conf/spec.yml
:
postgres_version: "31"
postgres_sha1: bbe4151f4000f349c2ffaf72412aac9cc0a3c741
New Features
- Migrated to Redis 5.0.
- Added a new
gbe recover
feature that runsbosh cloud-check
s (unattended with the-y
flag) on each target subsystem to re-create any missing VMs, which is very useful when recreating the Garden VM. - Add
cflinkuxfs3
while bumpingcf-deployment
from v5.4.0 to v6.10.0 (no breaking changes, though). - The
imported_vars
section now supports afrom: vars-file
feature that allows picking values from a variables file, located in theconf/
subdirectory of a subsystem. - Add the new
--compiled
flag, so thatgbe converge all --compiled
first upload any compiled releases from the local cache (as populated bygbe export
) to the BOSH Director.
Improvements
- Improved the Concourse subsystem in many ways, including a fixed CredHub integration, covered by Easy Foundry's exclusive smoke tests for Concourse. This also reflects in the GBE Downstream (“gbeds” for short) for a standalone Concourse: gstackio/concourse-gbeds.
- Reduce the memory used by log-cache.
- Fix issue with
gbe converge all
when only manifests are converged after converging manifest for one subsystem. - Fix several issues with
gbe converge list
. - Version updates, see below.
Breaking changes
- Switched to Redis 5.0 in this version. There are no guaranties that the in-place upgrade will occur smoothly.
- Moved
rabbitmq.<easyfoundry-domain-name>
management UI torabbitmq.sys.<easyfoundry-domain-name>
.
Notice
- This release ships Logsearch v209.1.0, in order to prepare migrating to v210.0.0 and above, that will ship ElasticSearch 6. This means that when migrating from prevision versions of Easy Foundry, you must deploy this version.
Components Versions
Component | New Version | Old Version |
---|---|---|
bosh-deployment | 2019-04-12 | v1.1.0+ (2018-10-26) |
BOSH | 268.7.0 | 268.2.0 |
CredHub CLI | 2.2.1 | 1.5.3 |
Xenial Stemcells | 250.38 (2019-02-13) | 97.32 (2018-11-08) |
Cloud Foundry deployment | 6.10.0 | 5.4.0 |
CF-RabbitMQ | 265.0.0 | 252.0.0 |
CF-Redis | 434.2.1 (Redis 5.0) | 434.0.22 (Redis 4.0) |
Concourse | 4.2.3 | 4.2.1 |
Træfik | 1.5.0 | 1.2.0 |
Kong | 1.2.0 | 1.1.0 |
Logsearch | 209.1.0 | 209.0.2 |
GBE & Easy Foundry v3.0.0
New Features
- Migrated to the latest Xenial stemcells v97.x.
- GBE can now converge a unique BOSH environment layout, where the BOSH server is separated from the Garden backend that runs the managed nodes. This is best practice in order to be able to update the BOSH server with no downtime. The technology behind this is the Shadow CPI. To adopt this setup (recommended), use the
ddbox-standalone-garden-env
(to be converged first) along withddbox-standalone-bosh-env
, instead of the singleddbox-env
. Themake base-env
target converges this for you. - Easy Foundry now comes with an out-of-the-box integration between
minio
andscality
S3 storages, theshield-v7
backup solution, and thecassandra
andredis
clusters. As long as you converge them in order, you'll get Cassandra and Redis automatically backuped, with the archives stored in Minio. - Experimental CockroachDB and Kong CE subsystems have been added.
- GBE downstream projects can now be created. These are GBE projects that track an upstream “distribution” GBE repository, adding their own customizations to it. This is still experimental at the moment, but a working example GBE Downstream (“gbeds” for short) and related instructions can be found at gstackio/concourse-gbeds.
- Subsystems can now define their own cloud configs or runtime configs. This should make subsystems more modular. Many subsystems still need to be refactored in order to adopt this new paradigm.
Improvements
- Deployment order now correctly takes dependencies into consideration, when running
gbe converge all
orgbe converge deployments
. This is also reflected in the experimentalmake all
target, that is approximately the same asgbe converge deployments
. - The
scality
S3 server subsystem now works properly. - Version updates, see below.
Fixes
- Fix
cf
smoke tests when behindtraefik
. - Fix the
gbe export
feature for exporting compiled BOSH Release. - Fix
concourse
sanity tests when nofly
CLI is downloaded yet - Reverted back to Docker release v30.1.0, due to stemcell incompatibility in newer v32.0.0.
Breaking changes
- The
cf
subsys has adopted thepxc-release
with no easy upgrade path. If you're upgrading from previous version, you'll have to enable the necessary transition ops files in an intermediarygbe converge
step by yourself.
Caveats
- The
tcp-router
in Cloud Foundry is producing a high number ofhaproxy
zombie processes.
Components Versions
Component | New Version | Old Version |
---|---|---|
bosh-deployment | 1.1.0+ (2018-10-26) | 1.0.0 (2018-06-22) |
BOSH | 268.2.0 | 266.3.0 |
BOSH CLI | 5.3.1 | 2.45.0 |
Stemcells | Xenial 97.32 (2018-11-08) | Trusty 3586.24 (2018-06-18) |
Cloud Foundry deployment | 5.4.0 | 1.40.0 |
Cloud Foundry CLI | 6.40.0 | 6.33.1 |
Concourse | 4.2.1 | 3.14.1 |
CF-MySQL | 36.15.0 | 36.14.0 |
CF-RabbitMQ | 252.0.0 | 247.0.0 |
CF-Redis | 434.0.22 | 431.0.0 |
CockroachDB | 4 | n/a |
Kong | 1.1.0 | n/a |
Logsearch | 209.0.2 | 209.0.0 |
Minio | 2018-10-25T01-27-03Z | 2018-06-08T03-49-38Z |
Prometheus | 23.3.0 | 23.0.0 |
Scality S3 server (Zenko CloudServer) | Beta7.2.0 | n/a |
SHIELD v7 | 7.0.8 | 7.0.4 |
SHIELD v8 | 8.0.16 | 8.0.10 |
GBE & Easy Foundry v2.2.0
New Features
- Add support for separate vars files, either secret or not. Exemples are provided here and here.
- Add a new
--manifest
option togbe converge
in order to only generate the deployment manifest and credentials. - Add a
-y
flag togbe converge
,gbe delete
andgbe recreate
, replacing the former-n
for non-interactive runs. - Allow
gbe converge
,gbe delete
andgbe recreate
to take many subsys as argument, which is very convenient to converge only a subset of Easy Foundry. - Add a new
postgres
subsys, based on the latest foked Dingo PostgreSQL, based on v0.10.2. - Add a basic
neo4j
subsys, based on the work-in-progress Neo4j BOSH Release.
Improvements
- Improved error reporting on missing config vars.
- Reduced persistent disks for Elasticsearch in the
logsearch
subsys. - Improved networking support, with idempotent iptables setup on the vbox host, improved setup of routes to ensure the deployed nodes are reachable, and a stub for
sshuttle
automation. - Have the Redis nodes belong a dedicated network, so that security groups can be as narrowed as possible.
- Automate SSH key setup when creating a
ddbox
-type environment. - Version updates, see below.
Fixes
- Don't always create heavy debug logs from
bosh create-env
on environment creation. Now a$GBE_DEBUG_LEVEL
(integer) controls this. - Fixed
gbe routes
on Linux. - Fixed the
minio
subsys with correct persistent disk type. - Fixed the
redis
subsys, properly adding a missing ops file in Git. - Fixed
gbe import
that was generating boggus compiled releases with no packages. - Fixed the
zzz-compiled-release-helper
subsys that is to be run beforegbe import
. - Add workaround for the
'Too many authentication failures
error when runninggbe up
.
Caveats
- Concourse deployment manifest and credentials must be generated before deploying the
cf
subsys. For this,gbe converge --manifest concourse
is convenient. - The
scality
subsys is not working yet in this version. - The
neo4j
subsys does not deploy yet a service broker for Cloud Foundry to present a Neo4j service in its marketplace. - The
postgres
subsys has a log rotation bug that can lead to full persistent disks.
Components Versions
Component | New Version | Old Version |
---|---|---|
bosh-deployment | 1.0.0 | n/a |
BOSH | 266.4.0 | 266.3.0 |
Cassandra | 8 | 6+ |
Dingo PostgreSQL | 0.10.2+ | n/a |
Træfik | 1.2.0 | 1.1.0 |
GBE v2.1.0
New Features
- Added a new
redis
subsystem, providing Redis 3.2.8 databases. The service broker either creates shared instances on the node where the broker is running, or either gives access to a dedicated Redis node. By default, 2 such dedicated nodes are provisioned. This can be increased easily. - Upgrade to the latest Cloud Foundry deployment version 1.40.0, which will be the last before version 2.0 is out.
Improvements
- The
cf
subsystem now runs its smoke tests once converged. - Improved inline help, adding details about
gbe converge list
andgbe delete list
. - Use the
-y
flag for non-interactive invocations ofgbe update
,gbe converge
,gbe recreate
, orgbe delete
, instead of-n
. - Accept many subsystems as arguments for
gbe converge
,gbe recreate
, orgbe delete
, which is very convenient for deploying a subset of Easy Foundry components. - Improve error handling and messages.
- Enable BOSH Backup Restore (BBR) in Cloud Foundry subsystem.
- Give higher precedence to deployment variables over imported variables, in subsystems of type
bosh-deployment
. - Version updates, see below.
Fixes
- Fixed network name in
cassandra
subsystem. - Favor “subsystem” (or “subsys”) term over “deployment” in the inline help, in order not to confuse GBE subsystems with BOSH deployments. This is an on-going effort though, as the “deployment” wording is still used in some places.
Components Versions
Component | New Version | Old Version |
---|---|---|
BOSH | 266.3.0 | 265.2.0 |
Stemcells | 3586.24 (2018-06-18) | 3541.24 (2018-05-07) |
Cloud Foundry deployment | 1.40.0 | 1.30.0 |
CF-MySQL | 36.14.0 | 36.13.0 |
CF-RabbitMQ | 247.0.0 (RabbitMQ 3.7) | 244.0.0 |
CF-Redis | 431.0.0 | n/a |
Concourse | 3.14.1 | 3.12.0 |
Prometheus | 23.0.0 | 22.0.2 |
Minio | 2018-06-08T03-49-38Z | 2018-04-27T23-33-52Z |
SHIELD v8 | 8.0.10 | 8.0.8 |
GBE v2.0.2
New Features
- A new
minio
deployment has been added. We plan to use it for Cloud Foundry blobstore and SHIELD backups storage. Adding a CF Service Broker to provision buckets for CF apps would be nice too.
Improvements
- Improved error reporting when making a typo on a subsys name, which is a common error! This applies to
gbe converge
,gbe recreate
orgbe delete
. - Better support and documentation for the “distant Virtualbox” use case. It's now proved to work fine, using the
sshuttle
tool to create a simple tunnel. - AZ placement for Cassandra nodes is now randomized, which for us better matches with what you would expect from such a deployment.
- Version updates, see below.
Fixes
- The local
~/.ssh/known_hosts
file is no more erroneously clobbered. - On macOS, the
sudo
password is no more required at everygbe converge
invocation.
Components Versions
Component | New Version | Old Version |
---|---|---|
Cloud Foundry deployment | 1.30.0 | 1.29.0 |
CF-MySQL | 36.13.0 | 36.12.0 |
CF Containers Broker | 1.0.3 | 1.0.1 |
Docker | 32.0.0 | 31.0.1 |
Stemcells | 3541.24 (2018-05-07) | 3541.12 (2018-04-07) |
Minio | 2018-04-27T23-33-52Z | n/a |
GBE v2.0.1
Security Fixes
This version addresses the MS-ISAC advisory number 2018-046, as advertised in PHP buildpack v4.3.53 release notes. You'll need to restage any PHP app you had staged with previous versions of the PHP buildpack.
Components Versions
Component | New Version | Old Version |
---|---|---|
Cloud Foundry deployment | 1.29.0 | 1.28.0 |
CF-RabbitMQ | 244.0.0 (RabbitMQ 3.7) | v243.0.0 |
GBE v2.0.0
This is a very good version of GBE.
With so many improvements compared to v1.0 !
New Features
-
Introduce the concept of modular bricks called “subsys“ (or subsystems), defined by a
spec.yml
file and specifyingfeatures/
which are operations files. Subsystems types are:bosh-environment
,cloud-config
,runtime-config
,bosh-deployment
. -
Subsystems can now declare many input resources (which are git repositories, usually pinned to specific revisions), which is very convenient to leverage operations files that come from 3rd party Git repositories.
-
Sybsystems declare
imported_vars
which are grabbed from other subsystems. This is very convenient to avoid copy/paste passwords from one deployment to another when no shared Bosh Link is available. Now stemcells versions are synchronized with this too. -
Imported credentials go to the subsystem's own
depl-creds.yml
credentials file. No credentials are ever included in rendered manifests (which are meant to be Git-trackable, so they must be free of any secrets). We stick here to the convention thatdepl-manifest.yml
files can be tracked in Git anddepl-creds.yml
not. -
Support for
pre-deploy
andpost-deploy
hook scripts has been added. Now service broker get registered to Cloud Foundry as an automated part ofgbe converge all
. This has proved being very convenient to develop new releases, where apre-deploy
hook rebuilds and uploads the release as part of thegbe converge
command. -
Added more sysbsystems:
logsearch
is BOSH 2.0 deployment for an ELK cluster,rabbitmq
is a resilient 2-nodes RabbitMQ cluster,traefik
is a Træfik reverse-proxy. -
Added
gbe cf
helper to download the Cloud Foundry CLI easily -
Fixed
gbe credhub
-
Bump
dnscontrol
to version0.2.3
-
More consistent UX. Now there is only
gbe
left. Support for separate helper scripts likecreate-env
ordeploy
has been dropped. Now Direnv is completely optional.
Components Versions
Component | New Version | Old Version |
---|---|---|
BOSH | 265.2.0 | 264.4.0 |
Stemcells | 3541.12 (2018-04-07) | 3468.11 (2017-11-22) |
Cloud Foundry deployment | 1.28.0 | 1.0.0 |
CF-MySQL | 36.12.0 | 36.9.0 |
Concourse | 3.12.0 | 3.6.0 |
CF-RabbitMQ | 243.0.0 (RabbitMQ 3.7) | n/a |
Logsearch | 209.0.0 | n/a |
Prometheus | 22.0.2 | 20.0.0 |
SHIELD v7 | 7.0.4 | n/a |
SHIELD v8 | 8.0.8 | n/a |
GBE v1.0.0
This is an intermediate version of GBE, with a full-featured central gbe
script. direnv
is made optional and the various helper scripts that require it are still fully supported in this version.
Many, many improvements are shipped in this version.
- Features UAA and Credhub in the BOSH environment.
- Support for importing and exporting compiled releases to accelerate deployment time.
- Doesn't expose the BOSH server to the internet.
- Provides helpers to establish a compatible SOCKS5 proxy
- Provides helpers to converge an external DNS with
dnscontrol
- Features more deployments:
shield
backups,prometheus
monitoring - Features more data-services:
cassandra
and Docker-based data services (Mysql 5.6, Postgres 9.6 and Redis 3.2) - Use of BOSH DNS in the Cloud Foundry deployment.
- Updated versions:
cf-deployment
v1.0.0, Concourse v3.6.0, Bosh v264.4.0, CF-MySQL v36.9.0 - Features
gbe
sub-commands likegbe bbl
,gbe terraform
,gbe bosh
andgbe dnscontrol
to locally install the compatible versions of the external utilities thatgbe
requires.
GBE v0.5.0
This is an early version of GBE, based on the initial ideas developed for a simple BOSH 2.0 framework.
- Massively based on
direnv
. - Allowing interaction with the BOSH environment (
create-env
anddelete-env
helper scripts) when in the base directory of the project. - Enabling interaction with the BOSH director when in the
deployments/
sub-directory. - Allowing interaction of cloud-config, runtime-config, and deployments when in their respective sub-directories.