This patch release of Lagoon fully reverts the changes introduced in #3133 and #3141
The Lagoon team comprehensively tested these changes prior to release. However, on release, it became apparent that a very small minority of users were using docker-compose.yml files that were incorrectly formatted, and would therefore not be able to be effectively read using our automated tools.
These errors range from full syntax errors (that would prevent docker-compose from working) to minor issues that would cause it to work with some limitations, to other use cases not envisioned in developing our features.
In order to provide the best possible Lagoon experience, this release reverts the new functionality and instead introduces a docker-compose validate step that will raise an issue in the build logs if Lagoon discovers one of these errors. We still intend to release the updated build deploy code in Lagoon v2.9.0 but intend to monitor builds in the meantime to ensure that we've captured a wide range of configurations
tags v2.8.2 and v.2.8.3 have not been converted into releases and can be safely skipped.
Full Changelog: v2.8.1...v2.8.4
This release is a hotfix release to remedy the issue raised in #3179, and a specific use case experienced when a user is a member of multiple projects. It has backported three PRs that will be released in Lagoon v2.9.0
If you are not utilising the LAGOON_BACKUP_DEV_RETENTION
and LAGOON_BACKUP_PR_RETENTION
variables to set alternate retention periods for development environments, you can skip this release and wait for v2.9.0 (released in the next day or so after this)
Changes in this release
- [BACKPORT] disable dev env retention policy adjustments @shreddedbacon (#3179)
- [BACKPORT] fix: check for task "complete" status, as succeeded is being deprecated @shreddedbacon (#3168)
- [BACKPORT] Improve project page performance @rocketeerbkw (#3160)
Full Changelog: v2.8.0...v2.8.1
In this release
This release contains a security fix for an opendistro index creation permission issue (see GHSA-7jj3-wwp7-989p)
This release also introduces a new set of logic for the build-deploy image, processing routes and ingresses via the new https://github.com/uselagoon/build-deploy-tool - gradually more of the logic will transfer across - see uselagoon/build-deploy-tool#27 for details
Additionally, the permissions needed to add environment variables to projects have been revised to match the use case, and can now be actioned by a user with "maintainer" permission, instead of the previous "owner".
This release is built on the https://github.com/uselagoon/lagoon-images/releases/tag/22.4.1 images
The lagoon-charts update will follow shortly.
What's Changed
- Security fix - Improper creation of OpenDistro/OpenSearch index role permissions by the Lagoon API
- fixup ingress and persistant storage for node-based tests by @tobybellwood in #3137
- add quiet to syft command and clean up setx by @tobybellwood in #3131
- update RBAC for project add env var to maintainer by @tobybellwood in #3075
- feat: use build tool to generate ingress templates by @shreddedbacon in #3133
- remove autoidler by @shreddedbacon in #3140
- feat: upgrade lagoon-linter v0.5.0 -> v0.7.0 by @smlx in #3138
- feat: use build tool to generate autogen routes by @shreddedbacon in #3141
- update build-deploy tool for autogen route improvements by @shreddedbacon in #3142
Full Changelog: v2.7.1...v2.8.0
This is a patch release to remedy a couple of issues found since releasing 2.7.0
What's Changed
- fix up counter check for dbaas provisioning by @shreddedbacon in #3127
- Set OpenDistro cluster permissions correcly and configure OpenSearch backend_roles by @tobybellwood in #3123
- more explicit controller Harbor robot actions by @tobybellwood in #3128
Full Changelog: v2.7.0...v2.7.1
This release is built on the https://github.com/uselagoon/lagoon-images/releases/tag/22.4.1 release
There are two feature updates in this release:
- Making tasks more like deployments - with similar progress stages for reporting and tracking
- Configurable backup schedules for non-production environments, with the ability to differentiate between PR and branch deployments. More information on how to configure them is in #3087
This release also remediates two earlier issues with S3 restore checks from v2.6.0 and some incorrect database migrations logic that caused the migrations not to run correctly.
What's Changed
- Fix the s3 restore check by @shreddedbacon in #3115
- Fix
add_ecdsa_ssh_key_types
migration by @rocketeerbkw in #3117 - Make tasks more like deployments by @shreddedbacon in #3114
- feature: configurable backup schedules and retention for development environments by @shreddedbacon in #3087
- fix errors in migrations sql by @tobybellwood in #3120
Full Changelog: v2.6.0...v2.7.0
Security FIx
This release of Lagoon patches a security vulnerability present in Lagoon v2.5.0 only. In this release, a service-api
client was added to Keycloak without a secret being automatically generated. This client isn't in a release yet (it is pre-work for the SSH portal coming shortly). THis v2.6.0 releae resolves this by automatically creating or rotating a secret. The corresponding charts release also allows for the definition of a secret, although this isn't supported in our version of keycloak yet.
If you are unable to upgrade to v2.6.0 immediately, you can log in to Keycloak, go to the service-api client, and click "Regenerate secret".
What's Changed
- fix api-db fix-permissions permissions to actually fix api-db permissions by @tobybellwood in #3081
- feature: add a timeout on the rollout status watch by @shreddedbacon in #3089
- Typo: Correct version number for Solr 8 image by @kasperg in #3054
- Typo fix by @mxr576 in #3091
- Update README.md by @AlannaBurke in #3084
- feature: add a failure notice message to pod rollout failures by @shreddedbacon in #3088
- Minor fixes to Logging and Contributing documentation by @smlx in #3079
- add rootless rsync commands to drush rsync task by @Schnitzel in #3080
- Adds ack for environments with no workflows by @bomoko in #3078
- Add example for pinning Node.js version in
php-cli
images by @rocketeerbkw in #3076 - refactor: capture errors for deploytargets by @shreddedbacon in #3090
- fix autogenerated urls to correctly truncate if they are too long by @shreddedbacon in #3098
- Add support in kubectl-build-deploy-dind for running rootless by @smlx in #2572
- strip acl param from multipart task file uploads by @shreddedbacon in #3097
- ECDSA ssh key type support by @cdchris12 in #3099
- Dep. trivy integration from core by @bomoko in #3083
- feat: validate TLS for all k8s API interactions by @smlx in #3107
- initial k8s install docs update by @tobybellwood in #3085
- check s3 object exists before generating signed url by @shreddedbacon in #3105
- Feature/confirmation text for custom tasks by @bomoko in #3094
- Fixing links. by @AlannaBurke in #3109
- Update upstream images and deprecate oc-build-deploy-dind by @tobybellwood in #3110
- Adds custom task argument documentation to docs by @bomoko in #3111
New Contributors
Full Changelog: v2.5.0...v2.6.0
This release is built on the https://github.com/uselagoon/lagoon-images/releases/tag/22.2.0 images
New in this release
There are three main features debuting in this release, two of which are still under development, but are in pre-release and in active use already
Bulk Deployments
This allows a Lagoon user to trigger the simultaneous deployment of multiple sites at once, and for those deployments to be automatically allocated to lagoon-remotes and stampede protection/QoS implemented in the remote-controller. These deployments come with additional updates to the UI, linking bulk deployments together, and providing an overview screen for easy tracking.
Insights (pre-release)
Insights is a remote-to-core system that collects data (currently SBOM and image data per service) from Lagoon Builds (into configMaps), and then transmits it back to a handler that stores that data into S3, and processes "key facts" into the API, stored against the environment. Additional functionality will be added to be able to analyse this data for vulnerabilities and inconsistencies, triggering alerts and data to the API. The key facts are still in development, but the underlying data model isn't expected to change.
Workflows (pre-release)
Workflows is an extension of the tasks system that allows more control over when tasks are run, what pre-conditions must exist. It's still in development, but is already in use.
Other updates
There are a number of other fixes in here, including improvements to DBaaS detection, configuration for single-node clusters, Kibana integration, rootless migration updates, GitLab MR labels, and some improvements to task logging, build logging and error tracking in deployments.
Deprecations and Updates
- A large amount of legacy (pre-RBAC) code has been removed - this was no longer functional, and was adjudged safe to be removed
- Kubernetes 1.22 comaptible updates have been made to the Ingresses created by Lagoon. More 1.22 work on other Lagoon aspects is also underway
- Alpine 3.15 has been rolled out where possible to Lagoon services.
What's Changed
- Feature/workflows by @bomoko in #2943
- updating advanced task resolver and permissions by @timclifford in #2955
- Workflows/add and update workflows ap ichanges by @timclifford in #2969
- Fix the check for mongodb-dbaas by @shreddedbacon in #3032
- fix: handle symlink to charts directory other than a local subdirectory by @smlx in #3043
- reduce logging verbosity some more by @shreddedbacon in #3037
- feat: add configuration-complete readiness signalling to keycloak by @smlx in #3042
- implement kubectl check for lagoon-build and add more tests by @tobybellwood in #3039
- check if scc is present and prevent patching the build pod by @shreddedbacon in #3040
- allow rwx volumes to be changed to rwo using a feature flag by @shreddedbacon in #3038
- Add cluster permission to allow downloading of reports from Kibana by @twardnw in #3034
- Adjust permissions on openshift:view by @shreddedbacon in #3031
- Upgrade upstream images to Alpine 3.15 by @tobybellwood in #3041
- networking.k8s.io Ingress deprecations by @tobybellwood in #2815
- Add support for ssh-portal / service-api by @smlx in #2941
- fix: improve rootless migration logic by @smlx in #3051
- Add additional information for failed pods during a build by @shreddedbacon in #3049
- Cleanup pre-rbac data by @rocketeerbkw in #2871
- Adding docker inspect insights gathering by @timclifford in #3033
- remove ssh_portal and tests for now by @tobybellwood in #3053
- fix: set the idling values in the build spec correctly by @shreddedbacon in #3055
- Feature: Bulk deployments by @shreddedbacon in #3046
- Adds summary functionality to environment facts by @bomoko in #3062
- Use actual Merge Request number for GitLab MRs by @tobybellwood in #3060
- Adds fact summary functionality to environment fact resolver by @bomoko in #3063
- environment_fact table updates - service column and unique constraint by @timclifford in #3064
- Feature/workflows by @bomoko in #3045
- fix workflows tests by @tobybellwood in #3071
- Fix up task logs for environment names with slash in the name by @shreddedbacon in #3068
- Fixes error with argument names in advanced tasks by @bomoko in #3070
- Removes logic error where workflows exit before processing all regist… by @bomoko in #3067
Full Changelog: v2.4.1...v2.5.0
This image is built on the https://github.com/uselagoon/lagoon-images/releases/tag/22.1.0 release
This release introduces a number of new and improved features:
- SBOM generation per-service into namespace ConfigMap
- Integration with the latest amazeeio/dbaas-operator - to dynamically check for presence of dbaas providers
- Incremental build log generation - logs are sent to S3 at a number of relevant build stages
- Collecting the pod logs from failed deployments, to help diagnose failures.
- Retries to skopeo docker image commands to overcome transient read/write issues
- UI updates to show DeployTarget configs, and expose metadata about lagoon-remote clusters as they pertain to environments.
- Addition of a python-persistent helm-chart
- Removal of the legacy billing code from the API
- Conversion of the stored API DB procedures into knex.
What's Changed (since 2.4.0)
- Fix docker-compose-build to execute checkDBaaSHealth by @twardnw in #3027
- feat: wrap sbom generation in feature flag by @shreddedbacon in #3028
- Zip and check size of SBOM configMap before posting it by @timclifford in #3025
What's Changed (since 2.3.0)
- Update docs about
monitoring-path
by @rocketeerbkw in #2976 - use request-timeout for storage calculator by @tobybellwood in #2986
- fix: resolver for environment openshift needs to do a project lookup by @shreddedbacon in #2973
- add idling values to the build spec for the controller to inject by @shreddedbacon in #2979
- Updating SSH docs to include SCP instructions. by @cdchris12 in #2996
- add retries to skopeo commands by @tobybellwood in #2977
- add Blackfire variables documentation to php variables by @Schnitzel in #2999
- Removing Billing Code from API by @justinlevi in #2837
- Collect pod logs for failed deployments by @shreddedbacon in #3011
- install latest fluent-plugins by @tobybellwood in #3009
- Support checking the dbaas-operator http endpoint for provider support by @shreddedbacon in #3007
- Some minor changes and typos by @timclifford in #2992
- Fix/advanced task permissions and queries by @bomoko in #2993
- Add image scanning and SBOM creation back to lagoon tag publishing by @tobybellwood in #3013
- Fix Storybook/Chromatic integration by @timclifford in #3014
- Resolving alertmanager config creation bug by @cdchris12 in #2974
- Additional kubernetes resource fields and supported UI changes by @shreddedbacon in #3010
- Add python-persistent helm chart. by @steveworley in #2989
- Add middleware to UI for security headers. by @steveworley in #2381
- Removes stored proc calls from resolvers by @bomoko in #3006
- Add scroll-to-top-bottom arrow to deployments page by @timclifford in #3019
- Assorted OpenDistro and OpenSearch fixes by @tobybellwood in #3017
- Custom task arguments by @bomoko in #2920
- Use local cached copy of svcat for oc image by @tobybellwood in #3020
- SBOM generation into configmap on build by @timclifford in #3012
- Fixes rogue semicolon in custom task creation by @bomoko in #3021
- add buildName label to sbom configMap by @tobybellwood in #3023
- remove Minishift/k3d from local dev setup by @tobybellwood in #3024
Full Changelog: v2.3.0...v2.4.1
The following changes are all incorporated in the 2.4.1 release - that release also contains two hotfixes to build-deploys that can cause build failures.
Use of this release may result in some mariadb services incorrectly being allocated container resources (mariadb-single) instead of the expected dbaas ones. In addition, sites that generated large ConfigMaps of their SBOM may have their builds incorrectly reported as "failing". Both these issues are resolved in v2.4.1
What's Changed
- Update docs about
monitoring-path
by @rocketeerbkw in #2976 - use request-timeout for storage calculator by @tobybellwood in #2986
- fix: resolver for environment openshift needs to do a project lookup by @shreddedbacon in #2973
- add idling values to the build spec for the controller to inject by @shreddedbacon in #2979
- Updating SSH docs to include SCP instructions. by @cdchris12 in #2996
- add retries to skopeo commands by @tobybellwood in #2977
- add Blackfire variables documentation to php variables by @Schnitzel in #2999
- Removing Billing Code from API by @justinlevi in #2837
- Collect pod logs for failed deployments by @shreddedbacon in #3011
- install latest fluent-plugins by @tobybellwood in #3009
- Support checking the dbaas-operator http endpoint for provider support by @shreddedbacon in #3007
- Some minor changes and typos by @timclifford in #2992
- Fix/advanced task permissions and queries by @bomoko in #2993
- Add image scanning and SBOM creation back to lagoon tag publishing by @tobybellwood in #3013
- Fix Storybook/Chromatic integration by @timclifford in #3014
- Resolving alertmanager config creation bug by @cdchris12 in #2974
- Additional kubernetes resource fields and supported UI changes by @shreddedbacon in #3010
- Add python-persistent helm chart. by @steveworley in #2989
- Add middleware to UI for security headers. by @steveworley in #2381
- Removes stored proc calls from resolvers by @bomoko in #3006
- Add scroll-to-top-bottom arrow to deployments page by @timclifford in #3019
- Assorted OpenDistro and OpenSearch fixes by @tobybellwood in #3017
- Custom task arguments by @bomoko in #2920
- Use local cached copy of svcat for oc image by @tobybellwood in #3020
- SBOM generation into configmap on build by @timclifford in #3012
- Fixes rogue semicolon in custom task creation by @bomoko in #3021
- add buildName label to sbom configMap by @tobybellwood in #3023
- remove Minishift/k3d from local dev setup by @tobybellwood in #3024
Full Changelog: v2.3.0...v2.4.0
This is the most recent scheduled release of Lagoon, built from the https://github.com/uselagoon/lagoon-images/releases/tag/21.12.1 images
There are three main items here:
-
Support for deifining services in routerPatterns (#2953) - this will allow users (particularly those with multi-clusters) to define their own router patterns. The Lagoon default is
${service}.${environment}.${project}.clusterURL
- but this can cause issues with some certificate authorities when used to secure Autogenerated routes. This PR allows the service, environment and project combination to be defined per project (or per cluster) - commonly to${service}-${environment}-${project}.clusterURL
-
Support for Routes defined via the API (#2940) - this will allow Administrators to override, or add routes to projects without the need for them to be added to the project's .lagoon.yml file. This is especially handy from a support point of view, as well as in Polysite or Multisite applications.
-
Images from previous deployments available as cache in the build step (#2919) - this exposes some new environment variables into the Lagoon Build that provide the image reference for the previous deployment's images. These can then be loaded into your dockerfile as a cache, especially useful for builds that have submodules. There is a brief example we use for testing at https://github.com/uselagoon/lagoon/blob/main/tests/files/image-cache/Dockerfile#L17 but we will publish more information shortly
Other smaller fixes include improved logic for Drush sql-dumps, log verbosity improvements, our documentation change, cronjob fixes, storage-calculator improvements and some improvements to docker-host management.
What's Changed
- fix: actually check the project defined routerpattern exists by @shreddedbacon in #2926
- Documentation Fixes - Make uselagoon/* images easier discoverable by @dasrecht in #2934
- add fallback true to docker_pull by @tobybellwood in #2942
- Refactor drupal tests by @tobybellwood in #2936
- wrap HELM_ARGUMENTS in setx by @tobybellwood in #2928
- Adds image cache images to build by @bomoko in #2919
- Ensure to only capture single digest by @tobybellwood in #2944
- Adds tests for cache and fixes env var format by @bomoko in #2945
- feat: support routes from the api in environment variables by @shreddedbacon in #2940
- Reduce concurrent tests per run - adding third suite temporarily by @tobybellwood in #2948
- Fix error running addKubernetes with empty id by @rocketeerbkw in #2929
- Passing no-tablespaces flag into drush sql-dump for task by @timclifford in #2939
- gitignore stern by @shreddedbacon in #2963
- fix: use the project name instead of the environment name by @shreddedbacon in #2954
- first batch of updated docker images by @tobybellwood in #2938
- change to mkdocs from gitbook by @tobybellwood in #2968
- feat: support for service being defined in routerPattern by @shreddedbacon in #2953
- Adding startingDeadlineSeconds to all cli native cronjob templates by @cdchris12 in #2831
- Refactor Storage Calculator to be more robust by @seanhamlin in #2947
- prune dangling images on docker-host automatically by @twardnw in #2967
- docker-host - added environment variable PRUNE_IMAGES_UNTIL to prune-… by @dasrecht in #2960
Full Changelog: v2.2.4...v2.3.0