No pod annotations can be added to a scheduled task

[zgassmann]

When launching a task from the GUI the pod gets the jobAnnotations defined either in the deployment properties or in the default task.platform.kubernetes.accounts.jobAnnotations.

When scheduling the task, the cronjob gets the jobAnnotations but not the pod of the task.
How can we add pod annotations to a scheduled task ?

[ilayaperumalg]

This is a new feature and we'll have this addressed in the upcoming release.

[zgassmann]

Thanks a lot for fixing this issue !

[ilayaperumalg]

@zgassmann Would you mind giving this a try on the 2.8.0-SNAPSHOT version of SCDF server? We plan to do the release early next month and it would be nice to get your feedback before. Thanks!

[zgassmann]

@ilayaperumalg, yes we would like to test it. At moment we are deploying with helm files using:

server:
// Bitnami Spring Cloud Dataflow Server image
// ref: https://hub.docker.com/r/bitnami/spring-cloud-dataflow/tags/

image:
registry: docker.io
repository: bitnami/spring-cloud-dataflow
tag: latest

How is the best way to test the 2.8.0-SNAPSHOT deploying with helm files in kubernetes ? Is there a repository I can reference ?

Thanks !

[ilayaperumalg]

Bitnami chart for 2.8.0 would be available only after we do the release of the same. So, you may want to try using the image from here

[zgassmann]

@ilayaperumalg, thanks you,
Meanwhile I deployed the 2.8.0-SNAPSHOT version on our test kubernetes environment.
I have one question, using the same helm files the new version is not enabling the connection to the PostgreSQL database but to the H2 database: Starting H2 Server with URL: jdbc:h2:tcp://localhost:19092/mem:dataflow

In the config map we have the data source defined, but it is not picked up in the new deployed version:

spring:
datasource:
url: 'jdbc:postgresql://sandbox-db-cluster.sandbox:5432/dataflow'
driverClassName: org.postgresql.Driver

Something has changed for that since 2.7.1 ?

[cppwfs]

Are you building locally? If so you will need to create your image using packeto. Installation instructions are here: https://paketo.io/docs/
And to build the image:

pack build \
   --path spring-cloud-dataflow-server/target/spring-cloud-dataflow-server-2.8.0-SNAPSHOT.jar \
   --builder gcr.io/paketo-buildpacks/builder:0.1.99-base \
   --env BP_JVM_VERSION=8 springcloud/spring-cloud-dataflow-server:2.8.0-SNAPSHOT

[zgassmann]

Hi @cppwfs,

thank you for your answer. No, I am not building locally, I just referenced the snapshot image in our helm files:

repository: springcloud/spring-cloud-dataflow-server
tag: 2.8.0-SNAPSHOT

instead of

repository: bitnami/spring-cloud-dataflow
tag: latest

The server is starting up. The difference to the 2.7 version is that it is using H2 instead of PostgreSQL.

I left our helm files the same (despite the image reference) including the datasource definition. Maybe I need to change the helm files for the 2.8 version so that the PostgreSQL datasource is used?

[zgassmann]

@ilayaperumalg, meanwhile I deployed the 2.8.0-SNAPSHOT version in our kubernetes cluster
and I can confirm that the podAnnotations now are working.
Thank you !

[ilayaperumalg]

@zgassmann Thank you for confirming!

[zgassmann]

Hi @ilayaperumalg, I have another question about the scheduling.
When we register a new version of an application, then a scheduled task still is using the version of the task app when it was scheduled. Launching the task manually uses the new docker image. Is there a possibility that a scheduled task is using the new registered task app version when its launched the next time ?

[cppwfs]

In this case SCDF, creates a cronjob for each schedule that is created. When you register a new version of the application, you will need to delete the schedule and recreate it, so that a new cronjob will be created using the new image.