not able to run task with java 11 / allow definition of environment variables for a task when launched on cloudfoundry

[imod]

Problem description:
Following scenario:

• I run SCDF on cloudfoundry.
• I define an app. (a Spring Boot Boot app requiring java 11)
• I define a task from the given app.
• I launch the task
• CF uses the java_buildpack to create the app container with java 8 (java 8 is the default)
• CF starts the app
• the app fails to start, because the wrong java version is used (8 instead of 11)

Solution description:
There should be a way to pass environment variables. To tell the java_buildpack to use the correct java version, one has to define the following environment variable (e.g. in the manifest.yml:

JBP_CONFIG_OPEN_JDK_JRE: '{jre: { version: 11.+ }}'

A possible solution could be additional params like this:
deployer.cloudfoundry.env.JBP_CONFIG_OPEN_JDK_JRE='{jre: { version: 11.+ }}'
deployer.cloudfoundry.env.XXXX=zzzzzzz
deployer.cloudfoundry.env.YYYY=wwwww

It is important to understand that defining only one environment variable might not be enough.

How ever, these environment variables must then be passt to cloudfoundry while deploying the app.

Additional context:
I also created an issue which describes that currently the buildpack parameter is not interpreted correctly: #3466

[sabbyanandan]

Hi, @imod. I had initially thought we could hack it by passing JBP_CONFIG_OPEN_JDK_JRE in the following ways, but I don't think they would take into account since this property is expected to be made available to the JBP way up in the app-push lifecycle. So, none of these options would work.

1. Setting "spring.cloud.dataflow.applicationProperties.task.JBP_CONFIG_OPEN_JDK_JRE": "'{ jre: { version: 11.+ }}'" as a flat global env-var for all the Tasks via SCDF.

2. Passing it as application/deployer property. [i.e., app.*.JBP_CONFIG_OPEN_JDK_JRE or deployer.*.cloudfoundry.JBP_CONFIG_OPEN_JDK_JRE along with deployer.*.cloudfoundry.useSpringApplicationJson=false].

Maybe @markpollack or @dturanski might have any other ideas to solve this. Let's see what they have to say.

[dturanski]

@sabbyanandan - I notice that the task launcher does not apply use-spring-application-json = false.

[dturanski]

This works for streams:

dataflow:>stream create ticktock --definition "time --JBP_CONFIG_OPEN_JDK_JRE='{jre: {version: 11.+}}' | log --JBP_CONFIG_OPEN_JDK_JRE='{jre: {version: 11.+}}'"
...
dataflow:>stream deploy ticktock --properties "deployer.*.cloudfoundry.use-spring-application-json=false"

Note, spaces after the :s . SAJ=false is also required as the build pack does not recognize SAJ during staging.

 2019-08-28T09:32:46.35-0400 [STG/0] OUT -----> Java Buildpack v4.16.1 | https://github.com/cloudfoundry/java-buildpack.git#e7cd0d7
2019-08-28T09:32:46.68-0400 [STG/0] OUT -----> Downloading Jvmkill Agent 1.16.0_RELEASE from https://java-buildpack.cloudfoundry.org/jvmkill/bionic/x86_64/jvmkill-1.16.0-RELEASE.so (0.1s)
2019-08-28T09:32:47.38-0400 [STG/0] OUT -----> Downloading Open Jdk JRE 11.0.4_11 from https://java-buildpack.cloudfoundry.org/openjdk/bionic/x86_64/openjdk-jre-11.0.4_11-bionic.tar.gz (0.6s)

[tzolov]

Resolved as part of spring-cloud/spring-cloud-deployer-cloudfoundry#308

[imod]

@tzolov do you provide SNAPSHOT builds of SCDF?

[ilayaperumalg]

@imod Given this is fixed in Spring Cloud Deployer CF 2.1.0.BUILD-SNAPSHOT, it is available in Spring snapshot repo and since the latest SCDF master has 2.1.0.M1 CF deployer, you can update the SCDF to use 2.1.0.BUILD-SNAPSHOT version of CF deployer and test. Let us know how it goes. Thanks!

[ilayaperumalg]

No, you are referring to the SCDF 2.2.0.RELEASE and what meant above is the CF deployer version. If you are building SCDF locally, then update your parent POM to use 2.1.0.BUILD-SNAPSHOT of spring-cloud-deployer-cloudfoundry and try this fix.

[imod]

@ilayaperumalg unfortunate it seems not to be enough to just define:

		<dependency>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-deployer-cloudfoundry</artifactId>
			<version>2.1.0.BUILD-SNAPSHOT</version>
		</dependency>

because a class seems to be missing now:

java.lang.NoClassDefFoundError: org/springframework/cloud/deployer/spi/cloudfoundry/CloudFoundry2630AndLaterTaskLauncher

[ilayaperumalg]

What version of SCDF do you use? You need to be on 2.2.2.BUILD-SNAPSHOT - the latest master that is.

[imod]

@ilayaperumalg I still did not manage to get the dependencies to work without getting:

Caused by: java.lang.NoClassDefFoundError: org/springframework/cloud/deployer/spi/cloudfoundry/CloudFoundry2630AndLaterTaskLauncher
	at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
	at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3167)
	at java.base/java.lang.Class.getDeclaredMethods(Class.java:2310)
	at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:489)

This is the source I use for testing: https://gist.github.com/imod/b7998965c31c4dd850999613c5db8460

[imod]

@tzolov if understand this all correctly, then this means it will not be possible to create task on CF that run with java11 and need parameters - as normal parameters will end up as environment variables and these are fixed on the created CF app/container. I think this quite a limitation of the current implementation and should be clearly documented.

[dturanski]

@imod - I'm not sure what you mean by normal parameters here. Typically task/job parameters are passed as command line arguments, specified as part of the task launch command. These can be different for every launch and will not be affected by this. SCDF sets certain app properties as environment variables and, by default, these are converted to JSON and set as the environment variable SPRING_APPLICATION_JSON. This is a standard Spring Boot property source, that will be parsed as JSON to bind all the properties. There may be some edge cases, such as property names containing property placeholders, in which this might make a difference but since these property values are already defined in the SAJ environment variable, they are static, will be applied to every task container and cannot be changed via the SCDF API without destroying the task. If you experience any specific problems related to this, please let us know.

[imod]

@dturanski indeed, I did mix up some stuff here - sorry for the noise!