Releases: spring-cloud/spring-cloud-dataflow
Spring Cloud Data Flow 2.11.3
Dependent Projects and Compatibility
| Component | Version |
|---|---|
| Spring Cloud Deployer | 2.9.3 |
| Spring Cloud Skipper | 2.11.3 |
| Spring Cloud Dataflow UI | 3.4.3 |
| Spring Cloud Dataflow | 2.11.3 |
Issues
- spring-cloud/spring-cloud-deployer#444 Update Kubernetes Support Matrix for 2.9.x
- spring-cloud/spring-cloud-dataflow-ui#1996 Update Task Execution list to use tasks/thinexecutions
- #5794 Fix code scanning alert - Spring-Kafka has Java Deserialization vulnerability When Improperly Configured
- #5783
--propertiesFileoption on scdf shell command "stream update" not actually available. - #5782 Fix the thin task execution page query for CTR status
- #5780 Vulnerabilities affecting Spring Cloud Dataflow dependencies
- #5778 Publish Skipper 2.11.x reference docs
- #5718 Unable to get list of task executions with a large amount of task execution rows
- #5695 Further performance improvements for job executions (for oracle, other dbs might vary)
- #5643 Address json-path CVE-2023-51074
- #5640 Re-enable SimpleJobServicePostgresTests
Spring Cloud Data Flow 2.11.2
Dependent Projects and Compatibility
| Component | Version |
|---|---|
| Spring Cloud Dataflow Build | 2.11.2 |
| Spring Cloud Dataflow Common | 2.11.2 |
| Spring Cloud Deployer | 2.9.2 |
| Spring Cloud Deployer Local | 2.9.2 |
| Spring Cloud Deployer CF | 2.9.2 |
| Spring Cloud Deployer K8S | 2.9.2 |
| Spring Cloud Common Security Config | 2.11.2 |
| Spring Cloud Skipper | 2.11.2 |
| Spring Cloud Dataflow UI | 3.4.2 |
| Spring Cloud Dataflow | 2.11.2 |
Issues
#5593 Update logback to handle CVE-2023-6378
spring-cloud/spring-cloud-deployer#407 CF Deployer needs to use LogCacheClient to retrieve logs
spring-cloud/spring-cloud-dataflow-ui#1974 Creating Schedule of Task prevents version property
spring-cloud/spring-cloud-dataflow-ui#1944 Task Execution Log doesn't scroll horizontally
#5615 Unable to sort tasks by Schema Target
#5611 Update to Spring Cloud 2021.0.9
#5609 Remove of JOB_CONFIGURATION_LOCATION broke some queries
#5591 The BOOT3 batch schema has old columns in table
#5576 Mariadb Schema needs to change BOOT3 Sequence tabels to sequences.
#5568 Schedule Task with Arguments and Properties does not work
#5564 Check dependencies for Spring Boot 2.7.18
#5561 Update Spring Boot 2.7.18
#5522 Update to Spring Boot 2.7.17
#5557 Step Context section is not populated on the UI
#5555 BOOT3_BATCH_JOB_EXECUTION should not contain JOB_CONFIGURATION_LOCATION
#5551 Batch Remote Parititioning does not pass Boot3 info to partitions
#5549 getJobExecutionsWithStepCountFilteredByTaskExecutionId in JdbcSearchableJobExecutionDao does not support task prefixes
#5548 Unable to configure Harbor as container registry
#5539 As a developer I need to run Oracle locally on my Mac ARM64
#5534 Task re-launch fails when using BOOT3 and Oracle
#5533 Add Boot3 Support for SimpleJobService
#5531 Unable to navigate through the job execution pages
#5524 Controller /jobs/thinexecutions is very slow with a large amount of job executions and no navigation possible oracle db
#5520 Stopping a batch with Spring Boot 3/Spring Batch 5 doesn't work properly.
#5518 Relaunch of a Spring Boot 3/Spring Batch 5 batch doesn't work properly.
#5507 Add commit info to the about controller
#5496 spring.cloud.dataflow.task.platform.kubernetes.accounts.[platformid].initContainer no longer injects the initContainer
#5484 Inconsistent data when consulting jobs/instances
#5464 Add documentation for launching multiple versions of a task.
#5408 Error during "Clean up all task/job executions"
Update
Kubernetes Deployment scripts updated.
Spring Cloud Data Flow 2.11.1
Notable changes in 2.11.1
- Updated versions and mitigations for CVEs.
- json-smart: 2.4.11
- nimbus-jose-jwt: 9.31
- snappy-java: 1.1.10.4
- commons-compress: 1.24.0
- Ensure TaskOperations launch API is backwards compatible.
- Fix issues in dependency managements of modules after creating mono-repo
Dependent Projects and Compatibility
| Component | Version |
|---|---|
| Spring Cloud Dataflow Build | 2.11.1 |
| Spring Cloud Dataflow Common | 2.11.1 |
| Spring Cloud Deployer | 2.9.1 |
| Spring Cloud Deployer Local | 2.9.1 |
| Spring Cloud Deployer CF | 2.9.1 |
| Spring Cloud Deployer K8S | 2.9.1 |
| Spring Cloud Common Security Config | 2.11.1 |
| Spring Cloud Skipper | 2.11.1 |
| Spring Cloud Dataflow UI | 3.4.1 |
| Spring Cloud Dataflow | 2.11.1 |
Issues
#5492 TaskOperations launch API is not backwards compatible
#5488 Update dependencies for latest CVEs
#5485 Add common-security-config modules to dependency management
Installer
The scripts for installing the Carvel package on Kubernetes and local development is attached below
Spring Cloud Data Flow 2.11.0
Noteworthy
Spring Boot 3 Applications
- Adds support for Spring Boot 3.x based stream applications.
- Adds support for Spring Cloud Task 3.x based task applications.
- Adds support for Spring Batch 5.x based batch applications.
More info can be found in the Boot 3 Appendix.
Library Updates
- Spring Boot 2.7.15
- Spring Cloud 2021.0.8
Kubernetes Update
- The long-awaited update of Kubernetes cronjobs from batch/v1beta -> batch/v1 which allows users to run Kubernetes >= 1.25.0.
Dependent Projects and Compatibility
| Component | Version |
|---|---|
| Spring Cloud Dataflow Build | 2.11.0 |
| Spring Cloud Dataflow Common | 2.11.0 |
| Spring Cloud Deployer | 2.9.0 |
| Spring Cloud Deployer Local | 2.9.0 |
| Spring Cloud Deployer CF | 2.9.0 |
| Spring Cloud Deployer K8S | 2.9.0 |
| Spring Cloud Common Security Config | 2.11.0 |
| Spring Cloud Skipper | 2.11.0 |
| Spring Cloud Dataflow UI | 3.4.0 |
| Spring Cloud Dataflow | 2.11.0 |
Issues
spring-cloud/spring-cloud-deployer#399 Duplicate Kubernetes pod ids when tasks are launched at the same time
spring-cloud/spring-cloud-dataflow-ui#1947 App registration details panel not updating when switch between boot versions
spring-cloud/spring-cloud-dataflow-ui#1946 Spring Boot 3 / Batch 5: Job execution is empty although shown in DB
#5467 Default version is always used to pick the table prefixes
#5459 The task configured as boot2 application cannot be executed in SCDF 2.11.0 RC1.
#5456 Error Value too long for column "AUDIT_DATA CHARACTER VARYING(4000)
#5452 Sensitive data still displayed in audit dashboard
#5450 As a user I can easily tell that this repo is no longer active and now lives in the mono repo.
#5446 Add tests to the spring-cloud-dataflow-aggregate-task module
#5443 [Docs] Missing task 3.x coordinates in ref guide
#5442 Add num days option to task cleanup
#5441 SCDF 2.11.0-RC1 composed task runner documentation
#5439 LOB-Elements broken after migration with PostgreSQLTextToOID
Spring Cloud Data Flow 2.10.3
Notable changes
Library updates
- Spring Boot 2.7.11
Dependent Projects and Compatibility
| Component | Version |
|---|---|
| Spring Cloud Dataflow Build | 2.10.3 |
| Spring Cloud Dataflow Common | 2.10.3 |
| Spring Cloud Deployer | 2.8.3 |
| Spring Cloud Deployer Local | 2.8.3 |
| Spring Cloud Deployer CF | 2.8.3 |
| Spring Cloud Deployer K8S | 2.8.3 |
| Spring Cloud Common Security Config | 1.8.3 |
| Spring Cloud Skipper | 2.9.3 |
| Spring Cloud Dataflow UI | 3.3.3 |
| Spring Cloud Dataflow | 2.10.3 |
CVE Fixes
#5329 Fix code scanning alert - Constructor Deserialization Remote Code Execution
#5303 Security issues in transitive dependency of spring-security-oauth2-client-5.4.2
#5302 Security issues in transitive dependency of spring-expression-5.2.11
#5301 Security issues in transitive dependency of spring-webmvc-5.3.25
#5300 Security issues in transitive dependency of json-smart-2.3
#5299 Security issues in transitive dependency of jettison-1.51
#5248 Misalignment of spring-security-oauth2-client versions in spring-cloud-dataflow-server module
Issues
#5325 Switch maven repos off of libs-xyz variants
#5321 default spring repo at local scdf is not available
#5317 Unregistering multiple applications doesn't work while unregistering single application works.
#5297 [2.10.x] Documentation cleanup
#5252 Simplify Carvel packages (Part 1)
#4791 Hibernate statistics are not exposed even if corresponding configuration are applied
spring-cloud/spring-cloud-deployer#381 (Backport) Respect initContainer from global properties
spring-cloud/spring-cloud-deployer#386 (Backport) Add pod-level 'terminationGracePeriodSeconds'
Spring Cloud Data Flow 2.10.2
Notable changes
Library updates
- Spring Boot 2.7.9
- Spring Cloud 2021.0.6
Dependent Projects and Compatibility
| Component | Version |
|---|---|
| Spring Cloud Dataflow Build | 2.10.2 |
| Spring Cloud Dataflow Common | 2.10.2 |
| Spring Cloud Deployer | 2.8.2 |
| Spring Cloud Deployer Local | 2.8.2 |
| Spring Cloud Deployer CF | 2.8.2 |
| Spring Cloud Deployer K8S | 2.8.2 |
| Spring Cloud Common Security Config | 1.8.2 |
| Spring Cloud Skipper | 2.9.2 |
| Spring Cloud Dataflow UI | 3.3.2 |
| Spring Cloud Dataflow | 2.10.2 |
Issues
#5232 Changed default repo.spring.io/snapshot to repo.spring.io/libs-snapshot
#5207 Removed invalid deprecation on StepExecutionHistory.getReadCount
#5246 Fixed by #5232
Spring Cloud Data Flow 2.10.1
Notable changes
Library updates
- Spring Boot
2.7.8 - Spring Framework
5.3.25 - Spring Shell
2.1.5
Kubernetes
- Added ability to fully configure pod and container security contexts.
- Container security context is propagated to init container and additional containers.
Dependent Projects and Compatibility
| Component | Version |
|---|---|
| Spring Cloud Dataflow Build | 2.10.1 |
| Spring Cloud Dataflow Common | 2.10.1 |
| Spring Cloud Deployer | 2.8.1 |
| Spring Cloud Deployer Local | 2.8.1 |
| Spring Cloud Deployer CF | 2.8.1 |
| Spring Cloud Deployer K8S | 2.8.1 |
| Spring Cloud Common Security Config | 1.8.1 |
| Spring Cloud Skipper | 2.9.1 |
| Spring Cloud Dataflow UI | 3.3.1 |
| Spring Cloud Dataflow | 2.10.1 |
Issues
spring-cloud/spring-cloud-dataflow-build#95 Update spring-shell 2.1.5
spring-cloud/spring-cloud-deployer-local#219 maxConcurrentExecutionsReached() has a race-condition where multiple instances can start before it returns a result
spring-cloud/spring-cloud-deployer-kubernetes#512 Add support to configure SecurityContext/allowPrivilegeEscalation for the deployed containers
spring-cloud/spring-cloud-dataflow-ui#1904 Task executions status label
#5200 Clarify docs on extending classpath
#5194 Remove workaround for missing pivotal-cloudfoundry-client-reactor
#5193 Remove all uses of repo.spring.io/release (main)
#5191 Update to Spring Boot 2.7.8
#5188 Flyway error migrating from SCDF 2.9.5 to 2.10.0
#5187 AppRegistryCommands missing ApplicationType completion
#5184 As user I want to set runAsGroup property in podSecurityContext
#5183 Task execustion view report wrong status
#5174 dataflow.spring.io Batch related documentation issues
#5172 spring-cloud-dataflow-shell fails to parse kebab-case arguments
#5171 Misalignment of spring-security-oauth2-client versions in spring-cloud-dataflow-server module
#5170 Spring Batch run in K8S pod in SCDF fails to load MariaDB driver class
#5164 MySQL 5.7 not working w/ SCDF 2.10 when using config server
#5163 Documentation for 2.10.0 local SCDF still points to 2.10.0-SNAPSHOT for manual downloads
#4971 Deprecate Count attribute for Step History
Spring Cloud Data Flow 2.10.0
Notable changes
Spring updates
Updates to the following versions:
- Spring Boot
2.7.6 - Spring Framework
5.3.24 - Spring Cloud
2021.0.5
Spring Shell
Spring Shell has been updated to 2.1.4.
Documentation Updates
Container images for ARM.
Spring Cloud Data Flow and Skipper, and Stream Application projects contain scripts to use in creating containers when running on an ARM platform. The Reference Guide has a section describing the use of the scripts.
Scripts for local Kuberenetes Development
We have added a collection of scripts to the Spring Cloud Data Flow that makes it easy to launch a local Kuberenetes cluster and install SCDF with MariaDB and RabbitMQ or Kafka. See the Reference Guide for more info.
Dependent Projects and Compatibility
| Component | Version |
|---|---|
| Spring Cloud Dataflow Build | 2.10.0 |
| Spring Cloud Dataflow Common | 2.10.0 |
| Spring Cloud Deployer | 2.8.0 |
| Spring Cloud Deployer Local | 2.8.0 |
| Spring Cloud Deployer CF | 2.8.0 |
| Spring Cloud Deployer K8S | 2.8.0 |
| Spring Cloud Common Security Config | 1.8.0 |
| Spring Cloud Skipper | 2.9.0 |
| Spring Cloud Dataflow UI | 3.3.0 |
| Spring Cloud Dataflow | 2.10.0 |
Issues
spring-cloud/spring-cloud-dataflow-build#93 Update spring-shell 2.1.4
spring-cloud/spring-cloud-deployer-kubernetes#503 Improve documentation on kuberenetes deployment properites
spring-cloud/spring-cloud-deployer-kubernetes#494 Support for concurrency policy of a Job Schedule
spring-cloud/spring-cloud-skipper#1068 Ensure reference doc is up-to-date and valid
#5161 Shell fails with stream deploy using properties
#5151 Review outstanding issues for dataflow.spring.io
#5113 Obtain OSM license for SCDF PRO
#5108 Support for TAS 3.0
#5083 Communicate best practice for extending the classpath of Dataflow/Skipper
#4946 Revisit shell changes
#4944 Arm64 docker images for Apple M1 usage
#4830 spring cloud dataflow launch task on other the kubernetes cluster
#4807 Ensure reference doc is up-to-date and valid
#4792 Update K8s compatibility in test-plan and docs
#4780 Add to docker-compose install docs info on env-vars and ssl.
#4779 Issue with MariaDB when using skipper-deployment.yaml
Spring Cloud Data Flow 2.10.0-RC2
Notable changes
Spring updates
Updates to the following versions:
- Spring Boot
2.7.5 - Spring Framework
5.3.23 - Spring Cloud
2021.0.5
NOTE: The 2.10.0-RC1 release was a partial release of the Spring Cloud Deployer artifacts only.
Dependent Projects and Compatibility
| Component | Version |
|---|---|
| Spring Cloud Dataflow Build | 2.10.0-RC2 |
| Spring Cloud Dataflow Common | 2.10.0-RC2 |
| Spring Cloud Deployer | 2.8.0-RC2 |
| Spring Cloud Deployer Local | 2.8.0-RC2 |
| Spring Cloud Deployer CF | 2.8.0-RC2 |
| Spring Cloud Deployer K8S | 2.8.0-RC2 |
| Spring Cloud Common Security Config | 1.8.0-RC2 |
| Spring Cloud Skipper | 2.9.0-RC2 |
| Spring Cloud Dataflow UI | 3.3.0-RC2 |
| Spring Cloud Dataflow | 2.10.0-RC2 |
Issues
spring-cloud/spring-cloud-deployer-kubernetes#511 Fabric8 version misalignment
#5125 Update Spring Boot 2.7.5 and Spring Cloud 2021.0.5
#5122 Adding index to TASK_EXECUTION_PARAMS table
#5111 Colon (port) in docker registry URI brokes detection of WWW-Authorization URL in DockerConfigJsonSecretToRegistryConfigurationConverter
#5077 Support for launching Boot 3.0 apps
#5043 The Rest API does not allow you to specify deployment variables
#4753 Set proper boot actuator endpoint configuration when deploying apps.
#4617 Error handling enhancement: StringIndexOutOfBoundsException when executing task in k8s for malformed task arguments
Spring Cloud Data Flow 2.10.0-M2
Notable changes
Spring updates
Updates to the following versions:
- Spring Boot
2.7.3 - Spring Framework
5.3.22 - Spring Cloud
2021.0.3
Spring Shell
Spring Shell has been updated to 2.1.0-M5.
Add back mysql Support
In M1 we removed support for mysql. This milestone restores support for MySQL 5.7+ using the MariaDB JDBC driver. See the reference docs for more details.
Dependent Projects and Compatibility
| Component | Version |
|---|---|
| Spring Cloud Dataflow Build | 2.10.0-M2 |
| Spring Cloud Dataflow Common | 2.10.0-M2 |
| Spring Cloud Deployer | 2.8.0-M2 |
| Spring Cloud Deployer Local | 2.8.0-M2 |
| Spring Cloud Deployer CF | 2.8.0-M2 |
| Spring Cloud Deployer K8S | 2.8.0-M2 |
| Spring Cloud Common Security Config | 1.8.0-M2 |
| Spring Cloud Skipper | 2.9.0-M2 |
| Spring Cloud Dataflow UI | 3.3.0-M2 |
| Spring Cloud Dataflow | 2.10.0-M2 |
Issues
spring-cloud/spring-cloud-dataflow-build#78 Update spring-shell 2.1.0-M5
spring-cloud/spring-cloud-dataflow-build#77 Manage db2 jdbc driver
spring-cloud/spring-cloud-dataflow-build#75 Update Boot 2.7.0
spring-cloud/spring-cloud-dataflow-build#73 Update boot 2.7.0-RC1
spring-cloud/spring-cloud-dataflow-build#68 Update Boot 2.6.6
spring-cloud/spring-cloud-dataflow-common#20 Add custom MySQL database type
spring-cloud/spring-cloud-dataflow-common#18 Spring Boot 2.7.0-RC1 / Flyway 8.5.9 broke MariaDB/MySQL on SCDF
spring-cloud/spring-cloud-deployer#356 Manage commons-io dependency
spring-cloud/spring-cloud-deployer#355 Update maven-wagon 3.5.2
spring-cloud/spring-cloud-deployer-kubernetes#491 NPE in instance status
spring-cloud/spring-cloud-common-security-config#93 Remove deprecated spring-security-oauth2
spring-cloud/spring-cloud-skipper#1057 Use SafeConstructor with SnakeYaml (PORT from 2.8.x)
spring-cloud/spring-cloud-skipper#1049 Package db2 jdbc driver
spring-cloud/spring-cloud-skipper#1047 Update boot 2.7.0-RC1
spring-cloud/spring-cloud-skipper#1044 Update pack 0.24.1
spring-cloud/spring-cloud-skipper#1042 Remove deprecated spring-security-oauth2.
spring-cloud/spring-cloud-skipper#1036 Skipper fails with oauth config
spring-cloud/spring-cloud-skipper#1034 Tests fail under Windows
spring-cloud/spring-cloud-dataflow-ui#1838 Enable e2e tests on CI pipeline
spring-cloud/spring-cloud-dataflow-ui#1826 Add status on task execution
spring-cloud/spring-cloud-dataflow-ui#1815 Not showing JOB details (batch Jobs, Job execution id) in Latest task execution
spring-cloud/spring-cloud-dataflow-ui#1798 Color status on import streams/tasks
spring-cloud/spring-cloud-dataflow-ui#1730 Can't remove a task app property
#5080 Ensure ATs are covering the proper databases/drivers
#5079 Update ref docs on supported DB and drivers
#5062 CVE-2022-31197 in pgjdbc transitive dependency (requires manual dependency override)
#5060 cannot set container name as task name when task launch/execution
#5057 Remove db2 and oracle drivers
#5056 Update to Boot 2.7.3
#5054 Upgrade mariadb driver to the 3.0.x series
#5052 Update Spring Boot 2.7.3 + Spring Cloud 2021.0.3
#5050 Use SafeConstructor with SnakeYaml (PORT from 2.9.x)
#5026 Increase Security with CSP Settings
#4926 Package db2 jdbc driver
#4918 Update Boot 2.7.0
#4915 Please don't update ubuntu packages within the docker-compose
#4914 Move DataFlowIT to spring-cloud-dataflow-acceptance-tests.
#4909 Shell fails with app info
#4908 Security issues in transitive dependency of jackson-databind-2.11.4 (SPRING-CLOUD-DATAFLOW 2.9.4)
#4907 security issues in transitive dependency of posgresql 42.2.25 (SPRING-CLOUD-DATAFLOW 2.9.4)
#4904 Update boot 2.7.0-RC1
#4888 Update pack 0.24.1
#4887 Add support for MySQL 8 in OSS
#4879 Add Back support for mysql 5.7
#4875 Remove deprecated spring-security-oauth2
#4869 Reference guide on launching a task from a stream is out of date.
#4863 Update reference guides to include new team members
#4861 NullPointer throw when authenticate via Bearer Token to active endpoint i.e. register application
#4854 Proposal for documenting client-server compatibility on documentation
#4795 Update spring-shell 2.1.x
#4744 Experiment with App SSO (for K8s) + Spring Authorization Server
#4682 Allow RestTemplate configure with DataFlowClientAutoConfiguration
#4603 Retest Azure AD configuration