k8s:watch broken; Service redeploy error since upgrade to Fabric8 Kubernetes Client

[rohanKanojia]

Description

k8s:watch goal doesn't seem to be working since upgrade to KubernetesClient. Right now when I'm running mvn k8s:watch, I'm getting patch exception related to Service:

eclipse-jkube-demo-project : $ mvn k8s:watch
[INFO] Scanning for projects...
[INFO] 
[INFO] ----------------------< meetup:random-generator >-----------------------
[INFO] Building random-generator 0.0.1
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] >>> kubernetes-maven-plugin:1.4.0-SNAPSHOT:watch (default-cli) > :deploy @ random-generator >>>
[INFO] 
[INFO] >>> kubernetes-maven-plugin:1.4.0-SNAPSHOT:deploy (deploy) > install @ random-generator >>>
[INFO] 
[INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) @ random-generator ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] Copying 1 resource
[INFO] Copying 0 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ random-generator ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources) @ random-generator ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] skip non existing resourceDirectory /home/rokumar/work/repos/jkube-testing/eclipse-jkube-demo-project/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ random-generator ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ random-generator ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running meetup.randomgenerator.RandomGeneratorApplicationTests
16:17:37.631 [main] DEBUG org.springframework.test.context.BootstrapUtils - Instantiating CacheAwareContextLoaderDelegate from class [org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate]
16:17:37.639 [main] DEBUG org.springframework.test.context.BootstrapUtils - Instantiating BootstrapContext using constructor [public org.springframework.test.context.support.DefaultBootstrapContext(java.lang.Class,org.springframework.test.context.CacheAwareContextLoaderDelegate)]
16:17:37.664 [main] DEBUG org.springframework.test.context.BootstrapUtils - Instantiating TestContextBootstrapper for test class [meetup.randomgenerator.RandomGeneratorApplicationTests] from class [org.springframework.boot.test.context.SpringBootTestContextBootstrapper]
16:17:37.672 [main] INFO org.springframework.boot.test.context.SpringBootTestContextBootstrapper - Neither @ContextConfiguration nor @ContextHierarchy found for test class [meetup.randomgenerator.RandomGeneratorApplicationTests], using SpringBootContextLoader
16:17:37.674 [main] DEBUG org.springframework.test.context.support.AbstractContextLoader - Did not detect default resource location for test class [meetup.randomgenerator.RandomGeneratorApplicationTests]: class path resource [meetup/randomgenerator/RandomGeneratorApplicationTests-context.xml] does not exist
16:17:37.675 [main] DEBUG org.springframework.test.context.support.AbstractContextLoader - Did not detect default resource location for test class [meetup.randomgenerator.RandomGeneratorApplicationTests]: class path resource [meetup/randomgenerator/RandomGeneratorApplicationTestsContext.groovy] does not exist
16:17:37.675 [main] INFO org.springframework.test.context.support.AbstractContextLoader - Could not detect default resource locations for test class [meetup.randomgenerator.RandomGeneratorApplicationTests]: no resource found for suffixes {-context.xml, Context.groovy}.
16:17:37.675 [main] INFO org.springframework.test.context.support.AnnotationConfigContextLoaderUtils - Could not detect default configuration classes for test class [meetup.randomgenerator.RandomGeneratorApplicationTests]: RandomGeneratorApplicationTests does not declare any static, non-private, non-final, nested classes annotated with @Configuration.
16:17:37.703 [main] DEBUG org.springframework.test.context.support.ActiveProfilesUtils - Could not find an 'annotation declaring class' for annotation type [org.springframework.test.context.ActiveProfiles] and class [meetup.randomgenerator.RandomGeneratorApplicationTests]
16:17:37.744 [main] DEBUG org.springframework.context.annotation.ClassPathScanningCandidateComponentProvider - Identified candidate component class: file [/home/rokumar/work/repos/jkube-testing/eclipse-jkube-demo-project/target/classes/meetup/randomgenerator/RandomGeneratorApplication.class]
16:17:37.751 [main] INFO org.springframework.boot.test.context.SpringBootTestContextBootstrapper - Found @SpringBootConfiguration meetup.randomgenerator.RandomGeneratorApplication for test class meetup.randomgenerator.RandomGeneratorApplicationTests
16:17:37.814 [main] DEBUG org.springframework.boot.test.context.SpringBootTestContextBootstrapper - @TestExecutionListeners is not present for class [meetup.randomgenerator.RandomGeneratorApplicationTests]: using defaults.
16:17:37.814 [main] INFO org.springframework.boot.test.context.SpringBootTestContextBootstrapper - Loaded default TestExecutionListener class names from location [META-INF/spring.factories]: [org.springframework.boot.test.mock.mockito.MockitoTestExecutionListener, org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener, org.springframework.boot.test.autoconfigure.restdocs.RestDocsTestExecutionListener, org.springframework.boot.test.autoconfigure.web.client.MockRestServiceServerResetTestExecutionListener, org.springframework.boot.test.autoconfigure.web.servlet.MockMvcPrintOnlyOnFailureTestExecutionListener, org.springframework.boot.test.autoconfigure.web.servlet.WebDriverTestExecutionListener, org.springframework.boot.test.autoconfigure.webservices.client.MockWebServiceServerTestExecutionListener, org.springframework.test.context.web.ServletTestExecutionListener, org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener, org.springframework.test.context.event.ApplicationEventsTestExecutionListener, org.springframework.test.context.support.DependencyInjectionTestExecutionListener, org.springframework.test.context.support.DirtiesContextTestExecutionListener, org.springframework.test.context.transaction.TransactionalTestExecutionListener, org.springframework.test.context.jdbc.SqlScriptsTestExecutionListener, org.springframework.test.context.event.EventPublishingTestExecutionListener]
16:17:37.825 [main] DEBUG org.springframework.boot.test.context.SpringBootTestContextBootstrapper - Skipping candidate TestExecutionListener [org.springframework.test.context.transaction.TransactionalTestExecutionListener] due to a missing dependency. Specify custom listener classes or make the default listener classes and their required dependencies available. Offending class: [org/springframework/transaction/TransactionDefinition]
16:17:37.825 [main] DEBUG org.springframework.boot.test.context.SpringBootTestContextBootstrapper - Skipping candidate TestExecutionListener [org.springframework.test.context.jdbc.SqlScriptsTestExecutionListener] due to a missing dependency. Specify custom listener classes or make the default listener classes and their required dependencies available. Offending class: [org/springframework/transaction/interceptor/TransactionAttribute]
16:17:37.825 [main] INFO org.springframework.boot.test.context.SpringBootTestContextBootstrapper - Using TestExecutionListeners: [org.springframework.test.context.web.ServletTestExecutionListener@54c62d71, org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener@65045a87, org.springframework.test.context.event.ApplicationEventsTestExecutionListener@47f4e407, org.springframework.boot.test.mock.mockito.MockitoTestExecutionListener@2d1dee39, org.springframework.boot.test.autoconfigure.SpringBootDependencyInjectionTestExecutionListener@48d5f34e, org.springframework.test.context.support.DirtiesContextTestExecutionListener@fc258b1, org.springframework.test.context.event.EventPublishingTestExecutionListener@6ff65192, org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener@2dd80673, org.springframework.boot.test.autoconfigure.restdocs.RestDocsTestExecutionListener@4af0df05, org.springframework.boot.test.autoconfigure.web.client.MockRestServiceServerResetTestExecutionListener@57ea113a, org.springframework.boot.test.autoconfigure.web.servlet.MockMvcPrintOnlyOnFailureTestExecutionListener@acdb094, org.springframework.boot.test.autoconfigure.web.servlet.WebDriverTestExecutionListener@674bd420, org.springframework.boot.test.autoconfigure.webservices.client.MockWebServiceServerTestExecutionListener@2b0f373b]
16:17:37.828 [main] DEBUG org.springframework.test.context.support.AbstractDirtiesContextTestExecutionListener - Before test class: context [DefaultTestContext@43b6123e testClass = RandomGeneratorApplicationTests, testInstance = [null], testMethod = [null], testException = [null], mergedContextConfiguration = [WebMergedContextConfiguration@260e86a1 testClass = RandomGeneratorApplicationTests, locations = '{}', classes = '{class meetup.randomgenerator.RandomGeneratorApplication}', contextInitializerClasses = '[]', activeProfiles = '{}', propertySourceLocations = '{}', propertySourceProperties = '{org.springframework.boot.test.context.SpringBootTestContextBootstrapper=true}', contextCustomizers = set[org.springframework.boot.test.context.filter.ExcludeFilterContextCustomizer@7dfb0c0f, org.springframework.boot.test.json.DuplicateJsonObjectContextCustomizerFactory$DuplicateJsonObjectContextCustomizer@1f9e9475, org.springframework.boot.test.mock.mockito.MockitoContextCustomizer@0, org.springframework.boot.test.web.client.TestRestTemplateContextCustomizer@23941fb4, org.springframework.boot.test.autoconfigure.actuate.metrics.MetricsExportContextCustomizerFactory$DisableMetricExportContextCustomizer@6e2aa843, org.springframework.boot.test.autoconfigure.properties.PropertyMappingContextCustomizer@0, org.springframework.boot.test.autoconfigure.web.servlet.WebDriverContextCustomizerFactory$Customizer@20bd8be5, org.springframework.boot.test.context.SpringBootTestArgs@1, org.springframework.boot.test.context.SpringBootTestWebEnvironment@16e7dcfd], resourceBasePath = 'src/main/webapp', contextLoader = 'org.springframework.boot.test.context.SpringBootContextLoader', parent = [null]], attributes = map['org.springframework.test.context.web.ServletTestExecutionListener.activateListener' -> true]], class annotated with @DirtiesContext [false] with mode [null].
16:17:37.860 [main] DEBUG org.springframework.test.context.support.TestPropertySourceUtils - Adding inlined properties to environment: {spring.jmx.enabled=false, org.springframework.boot.test.context.SpringBootTestContextBootstrapper=true}

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.5.1)

2021-07-02 16:17:38.108  INFO 45595 --- [           main] m.r.RandomGeneratorApplicationTests      : Starting RandomGeneratorApplicationTests using Java 11.0.11 on fedora with PID 45595 (started by rokumar in /home/rokumar/work/repos/jkube-testing/eclipse-jkube-demo-project)
2021-07-02 16:17:38.110  INFO 45595 --- [           main] m.r.RandomGeneratorApplicationTests      : No active profile set, falling back to default profiles: default
2021-07-02 16:17:39.990  INFO 45595 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 1 endpoint(s) beneath base path '/actuator'
2021-07-02 16:17:40.046  INFO 45595 --- [           main] m.r.RandomGeneratorApplicationTests      : Started RandomGeneratorApplicationTests in 2.18 seconds (JVM running for 2.819)
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.8 s - in meetup.randomgenerator.RandomGeneratorApplicationTests
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- maven-jar-plugin:3.2.0:jar (default-jar) @ random-generator ---
[INFO] Building jar: /home/rokumar/work/repos/jkube-testing/eclipse-jkube-demo-project/target/random-generator-0.0.1.jar
[INFO] 
[INFO] --- spring-boot-maven-plugin:2.5.1:repackage (repackage) @ random-generator ---
[INFO] Replacing main artifact with repackaged archive
[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ random-generator ---
[INFO] Installing /home/rokumar/work/repos/jkube-testing/eclipse-jkube-demo-project/target/random-generator-0.0.1.jar to /home/rokumar/.m2/repository/meetup/random-generator/0.0.1/random-generator-0.0.1.jar
[INFO] Installing /home/rokumar/work/repos/jkube-testing/eclipse-jkube-demo-project/pom.xml to /home/rokumar/.m2/repository/meetup/random-generator/0.0.1/random-generator-0.0.1.pom
[INFO] 
[INFO] <<< kubernetes-maven-plugin:1.4.0-SNAPSHOT:deploy (deploy) < install @ random-generator <<<
[INFO] 
[INFO] 
[INFO] --- kubernetes-maven-plugin:1.4.0-SNAPSHOT:deploy (deploy) @ random-generator ---
[INFO] k8s: Using Kubernetes at https://192.168.39.43:8443/ in namespace default with manifest /home/rokumar/work/repos/jkube-testing/eclipse-jkube-demo-project/target/classes/META-INF/jkube/kubernetes.yml 
[INFO] k8s: Updating Service from kubernetes.yml
[ERROR] k8s: Failed to update Service from kubernetes.yml. io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: PATCH at: https://192.168.39.43:8443/api/v1/namespaces/default/services/random-generator. Message: Service "random-generator" is invalid: spec.clusterIPs[0]: Invalid value: []string(nil): primary clusterIP can not be unset. Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=spec.clusterIPs[0], message=Invalid value: []string(nil): primary clusterIP can not be unset, reason=FieldValueInvalid, additionalProperties={})], group=null, kind=Service, name=random-generator, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Service "random-generator" is invalid: spec.clusterIPs[0]: Invalid value: []string(nil): primary clusterIP can not be unset, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).. Service(apiVersion=v1, kind=Service, metadata=ObjectMeta(annotations={prometheus.io/path=/metrics, jkube.io/git-branch=master, prometheus.io/port=9779, jkube.io/scm-url=https://github.com/spring-projects/spring-boot/random-generator, prometheus.io/scrape=true, jkube.io/git-url=git@github.com:rohanKanojia/eclipse-jkube-demo-project.git, jkube.io/scm-tag=HEAD, jkube.io/git-commit=7be4f613695f489ff088bf49a943032e6fe7a79e}, clusterName=null, creationTimestamp=null, deletionGracePeriodSeconds=null, deletionTimestamp=null, finalizers=[], generateName=null, generation=null, labels={expose=true, app=random-generator, provider=jkube, version=0.0.1, group=meetup}, managedFields=[], name=random-generator, namespace=null, ownerReferences=[], resourceVersion=38573, selfLink=null, uid=null, additionalProperties={}), spec=ServiceSpec(allocateLoadBalancerNodePorts=null, clusterIP=null, clusterIPs=[], externalIPs=[], externalName=null, externalTrafficPolicy=null, healthCheckNodePort=null, internalTrafficPolicy=null, ipFamilies=[], ipFamilyPolicy=null, loadBalancerClass=null, loadBalancerIP=null, loadBalancerSourceRanges=[], ports=[ServicePort(appProtocol=null, name=http, nodePort=null, port=8080, protocol=TCP, targetPort=IntOrString(IntVal=8080, Kind=0, StrVal=null, additionalProperties={}), additionalProperties={})], publishNotReadyAddresses=null, selector={app=random-generator, provider=jkube, group=meetup}, sessionAffinity=null, sessionAffinityConfig=null, topologyKeys=[], type=NodePort, additionalProperties={}), status=null, additionalProperties={}): io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: PATCH at: https://192.168.39.43:8443/api/v1/namespaces/default/services/random-generator. Message: Service "random-generator" is invalid: spec.clusterIPs[0]: Invalid value: []string(nil): primary clusterIP can not be unset. Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=spec.clusterIPs[0], message=Invalid value: []string(nil): primary clusterIP can not be unset, reason=FieldValueInvalid, additionalProperties={})], group=null, kind=Service, name=random-generator, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Service "random-generator" is invalid: spec.clusterIPs[0]: Invalid value: []string(nil): primary clusterIP can not be unset, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  6.444 s
[INFO] Finished at: 2021-07-02T16:17:42+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.eclipse.jkube:kubernetes-maven-plugin:1.4.0-SNAPSHOT:deploy (deploy) on project random-generator: Failed to update Service from kubernetes.yml. io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: PATCH at: https://192.168.39.43:8443/api/v1/namespaces/default/services/random-generator. Message: Service "random-generator" is invalid: spec.clusterIPs[0]: Invalid value: []string(nil): primary clusterIP can not be unset. Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=spec.clusterIPs[0], message=Invalid value: []string(nil): primary clusterIP can not be unset, reason=FieldValueInvalid, additionalProperties={})], group=null, kind=Service, name=random-generator, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Service "random-generator" is invalid: spec.clusterIPs[0]: Invalid value: []string(nil): primary clusterIP can not be unset, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).. Service(apiVersion=v1, kind=Service, metadata=ObjectMeta(annotations={prometheus.io/path=/metrics, jkube.io/git-branch=master, prometheus.io/port=9779, jkube.io/scm-url=https://github.com/spring-projects/spring-boot/random-generator, prometheus.io/scrape=true, jkube.io/git-url=git@github.com:rohanKanojia/eclipse-jkube-demo-project.git, jkube.io/scm-tag=HEAD, jkube.io/git-commit=7be4f613695f489ff088bf49a943032e6fe7a79e}, clusterName=null, creationTimestamp=null, deletionGracePeriodSeconds=null, deletionTimestamp=null, finalizers=[], generateName=null, generation=null, labels={expose=true, app=random-generator, provider=jkube, version=0.0.1, group=meetup}, managedFields=[], name=random-generator, namespace=null, ownerReferences=[], resourceVersion=38573, selfLink=null, uid=null, additionalProperties={}), spec=ServiceSpec(allocateLoadBalancerNodePorts=null, clusterIP=null, clusterIPs=[], externalIPs=[], externalName=null, externalTrafficPolicy=null, healthCheckNodePort=null, internalTrafficPolicy=null, ipFamilies=[], ipFamilyPolicy=null, loadBalancerClass=null, loadBalancerIP=null, loadBalancerSourceRanges=[], ports=[ServicePort(appProtocol=null, name=http, nodePort=null, port=8080, protocol=TCP, targetPort=IntOrString(IntVal=8080, Kind=0, StrVal=null, additionalProperties={}), additionalProperties={})], publishNotReadyAddresses=null, selector={app=random-generator, provider=jkube, group=meetup}, sessionAffinity=null, sessionAffinityConfig=null, topologyKeys=[], type=NodePort, additionalProperties={}), status=null, additionalProperties={}) -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

This is happening only on master, v1.3.0 is not affected with this issue

Info

• Eclipse JKube version : master
• Maven version (mvn -v) :

Apache Maven 3.8.1 (05c21c65bdfed0f71a2f2ada8b84da59348c4c5d)
Maven home: /opt/apache-maven
Java version: 11.0.11, vendor: Oracle Corporation, runtime: /usr/java/jdk-11.0.11
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.12.9-300.fc34.x86_64", arch: "amd64", family: "unix"

• Kubernetes / Red Hat OpenShift setup and version :

• If it's a bug, how to reproduce : run mvn k8s:watch on spring-boot sample with 1.4.0-SNAPSHOT version

• If it's a feature request, what is your use case :

• Sample Reproducer Project : [GitHub Clone URL]

[manusa]

Does this relate to what we talked about this morning?

[rohanKanojia]

Yes, this is the issue I've created for the additional ApplyService changed I had done in #748

[manusa]

Caused by: (check this discussion fabric8io/kubernetes-client#3221 (review))