apiVersion is ignored by resource and apply goals #2219

tipame · 2023-06-20T05:59:54Z

Describe the bug

Hello, in my project i have XML configuration with custom resurce:

apiVersion: my.unique.crd/v1alfa1
kind: Template

Also i have custom mapping definition:

<mapping>
  <kind>Template</kind>
  <filenameTypes>my_template</filenameTypes>
</mapping>

As you can see here - there is no way to specify apiVersion in mapping definition - so we have conflict with default Template.
Executing resource goal result to creating ***-template.yaml but not a ***-my_template.yaml (apiVersion property present in generated yaml).
Executing apply goal - ignores ***-template.yaml (no object created in kubernetes).

Executing apply with kubectl work fine (object successfully creates):
kubectl apply -f ***-template.yaml

Eclipse JKube version

1.13.1

Component

Kubernetes Maven Plugin

Apache Maven version

3.8.5

Gradle version

None

Steps to reproduce

Create CDR with kind Template (with your unique group):

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.10.0
  creationTimestamp: null
  name: templates.my.unique.group
spec:
  group: my.unique.group
  names:
    kind: Template
    listKind: TemplateList
    plural: templates
    singular: template
  scope: Namespaced
  versions:
  - name: v1alpha1
    schema:
      openAPIV3Schema:
        properties:
          apiVersion:
            description: 'APIVersion defines the versioned schema of this representation
              of an object. Servers should convert recognized schemas to the latest
              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
            type: string
          kind:
            description: 'Kind is a string value representing the REST resource this
              object represents. Servers may infer this from the endpoint the client
              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
            type: string
          metadata:
            type: object
          spec:
            description: TemplateSpec defines the desired state of Template
            properties:
              apiVersion:
                default: v1
                type: string
              data:
                type: string
              kind:
                default: Pod
                type: string
            required:
            - data
            type: object
          status:
            description: TemplateStatus defines the observed state of Template
            properties:
              conditions:
                items:
                  properties:
                    lastTransitionTime:
                      format: date-time
                      type: string
                    message:
                      type: string
                    status:
                      type: string
                    type:
                      type: string
                  required:
                  - status
                  - type
                  type: object
                type: array
              phase:
                type: string
            type: object
        type: object
    served: true
    storage: true
    subresources:
      status: {}

Create XML configuration for custom Template:

apiVersion: my.unique.group/v1alpha1
kind: Template
metadata:
  name: my-example
  namespace: my-namespace
spec:
data: |-
    spec:    
      containers:
        - name: my-container

Add custom mapping definition in pluging configuration:

<mappings>
    <mapping>
        <kind>Template</kind>
        <filenameTypes>my_template</filenameTypes>
    </mapping>    
</mappings>

k8s:resource produce my-example-template.yaml
k8s:apply ignores my-example-template.yaml (don't know is it right behavior for v1 Template)

Expected behavior

Expect plugin respect apiVersion and distinguish differend resources with same kind name.
Expect k8s:resource produce my-example-my_template.yaml
Expect k8s:apply successfuly applies my configuration my-example-my_template.yaml (object created in kubernetes).

Runtime

Kubernetes (vanilla)

Kubernetes API Server version

1.25.3

Environment

Windows

Eclipse JKube Logs

No response

Sample Reproducer Project

No response

Additional context

No response

The text was updated successfully, but these errors were encountered:

rohanKanojia · 2023-06-20T06:02:31Z

@tipame : ~~Are you using OpenShift templates in your project?~~

Sorry, I got confused with your CustomResourceDefinition naming.

Could you please share directory contents of fragment folder? I don't think you need to provide any additional configuration for CustomResources. Our E2E test with CRD seems to be passing.

tipame · 2023-06-20T06:23:19Z

@rohanKanojia you mean to attach generated resources?
jkube.zip

rohanKanojia · 2023-06-20T06:27:04Z

@tipame : No, I didn't mean that. Is it possible to share a simple reproducer project?

rohanKanojia · 2023-06-20T07:27:18Z

@tipame : I tried reproducing your issue but could not reproduce it. I copied the CustomResourceDefinition and CustomResource fragments into src/main/jkube folder unchanged. For me, CustomResourceDefinition and CustomResource seem to get applied okay. Could you please check my reproducer project to see if I missed something here jkube-customresource-fragments?

tipame · 2023-06-20T07:42:21Z

@rohanKanojia

You have no file mapping for custom Template. Actualy there is no way to set mapping in this case - cause thre is no tag for apiVersion (only kind and filenameTypes).
I have no errors applying config, but resource with "kind: Template" simply ignored. In maven log there is no entry like:
Created Custom Resource: my.unique.group/v1alpha1#Template my-namespace/my-example
As a result no object is created in kubernetes

Also should mention: work fine with other CRD (with kind: Trigger for example)

rohanKanojia · 2023-06-20T07:50:08Z

I'm not able to understand why you need to add a mapping. I tried running k8s:resource k8s:apply and I was able to see an entry like this:

[INFO] --- kubernetes-maven-plugin:1.13.1:apply (default-cli) @ jkube-customresource-fragments ---
[INFO] k8s: Using Kubernetes at https://192.168.39.103:8443/ in namespace null with manifest /home/rokumar/work/repos/jkube-testing/jkube-customresource-fragments/target/classes/META-INF/jkube/kubernetes.yml 
[INFO] k8s: Creating a Service from kubernetes.yml namespace default name jkube-customresource-fragments
[INFO] k8s: Created Service: target/jkube/applyJson/default/service-jkube-customresource-fragments-2.json
[INFO] k8s: Creating a Custom Resource Definition from kubernetes.yml name templates.my.unique.group
[INFO] k8s: Created Custom Resource Definition result: templates.my.unique.group
[INFO] k8s: Creating a Deployment from kubernetes.yml namespace default name jkube-customresource-fragments
[INFO] k8s: Created Deployment: target/jkube/applyJson/default/deployment-jkube-customresource-fragments-2.json
[WARNING] The client is using resource type 'templates' with unstable version 'v1alpha1'
[INFO] k8s: Created Custom Resource: my.unique.group/v1alpha1#Template my-namespace/my-example
[INFO] k8s: HINT: Use the command `kubectl get pods -w` to watch your pods start up
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  7.229 s
[INFO] Finished at: 2023-06-20T13:18:40+05:30
[INFO] ------------------------------------------------------------------------
jkube-customresource-fragments : $ kubectl get template -n my-namespace
NAME         AGE
my-example   56s
jkube-customresource-fragments : $

rohanKanojia · 2023-06-20T07:52:47Z

Could your issue be due to your mapping colliding with default mapping for OpenShift Template? See kind-filename-type-mapping-default.adoc

tipame · 2023-06-20T08:25:41Z

Main bug (apply goal) seams to be fixed in 1.13.1 (my plugin version was 1.6.0). Now resource is aplied.
About maping:
Imagine we have two CRD:

apiVersion: my.unique1.crd/v1alfa1
kind: Template
apiVersion: my.unique2.crd/v1alfa1
kind: Template

There is no way to set two differend mapings for this resources.
In my case it colliding with default mapping (as you mention).

manusa · 2023-06-20T08:28:50Z

Maybe you don't need a mapping at all. Just provide the name without the -$mapping.yaml suffix.

tipame · 2023-06-20T08:36:17Z

Maybe you don't need a mapping at all. Just provide the name without the -$mapping.yaml suffix.

all files in my project have names like: xxx-yyy-$mapping.yaml
renaming it to xxx-yyy.yaml result to error:
Unknown type 'yyy' for file xxx-yyy.yaml. Must be one of : pr, lr, pv, project, replicaset, cronjob

rohanKanojia · 2023-06-20T09:23:11Z

I think CustomResource fragments need to have -cr.yaml suffix

manusa · 2023-06-20T09:29:54Z

I think CustomResource fragments need to have -cr.yaml suffix

Forgot about that one :(. This is definitely something to improve.

Then having xxx-yyy-cr.yaml should work without the need for a mapping 🤞 .

tipame · 2023-06-20T10:59:49Z

I think CustomResource fragments need to have -cr.yaml suffix

Forgot about that one :(. This is definitely something to improve.

Then having xxx-yyy-cr.yaml should work without the need for a mapping 🤞 .

Yep "cr" suffix works! Thank you. Have no more question.

rohanKanojia · 2023-06-20T11:32:18Z

@tipame : Do you have some feedback on how we can improve this in documentation?

tipame · 2023-06-20T12:58:11Z

Back agan: "cr" = ClusterRole )))
If i use "cr" suffix without setting kind property in XML it's generated to ClusterRole...

In my opinion - there was a future to auto complete XML configuration base on filenaming:
I can ommit "apiVestion", "kind" and "name" properties and they should be generated based on mapings.
Since i can't set apiVersion in custom maping -> it always generated to "v1" (not what i want).

manusa · 2023-06-20T13:42:58Z

This is definitely a bug we need to tackle ASAP.

manusa · 2023-07-12T09:35:19Z

Hi @tipame
The current 1.14-SNAPSHOT should contain the fixes to allow for all of your expected behaviors.

Could you please try out the snapshot and confirm that everything is working for you as expected?

tipame · 2023-07-17T04:26:25Z

Hello @manusa, how can i specify apiVersion in a fixed branch?

rohanKanojia · 2023-07-17T06:33:36Z

@tipame : Marc is asking you to do this:

Enable pulling jkube snapshot repositories in pom:

  <pluginRepositories>
    <pluginRepository>
      <id>oss.sonatype.org</id>
      <url>https://oss.sonatype.org/content/repositories/snapshots</url>
      <snapshots>
        <enabled>true</enabled>
        <updatePolicy>always</updatePolicy>
      </snapshots>
    </pluginRepository>
  </pluginRepositories>

Update OpenShift Maven Plugin to use 1.14-SNAPSHOT:

            <plugin>
                <groupId>org.eclipse.jkube</groupId>
                <artifactId>openshift-maven-plugin</artifactId>
                <version>1.14-SNAPSHOT</version>
            </plugin>

Test whether you get desired behavior by updating version

tipame · 2023-07-17T08:01:57Z

Question about apiVersion (see isue description), not about maven.
Expect something like:

<mapping>
  <apiVersion>my.unique.crd/v1alfa1</apiVersion>
  <kind>Template</kind>
  <filenameTypes>my_template</filenameTypes>
</mapping>

manusa · 2023-07-17T08:26:24Z

Question about apiVersion (see isue description), not about maven. Expect something like:
<mapping>
  <apiVersion>my.unique.crd/v1alfa1</apiVersion>
  <kind>Template</kind>
  <filenameTypes>my_template</filenameTypes>
</mapping>

This is not possible at the moment. I did see it as something that would be useful when implementing the IT tests (https://github.com/eclipse/jkube/tree/e116ca23a6526a5498e38b20c55f4c755339c055/gradle-plugin/it/src/it/fragments-custom-mapping).

I think we can definitely add this since it makes much sense when working with CRs and other APIs.

tipame · 2023-07-17T08:47:46Z

So i do not undestand what should i test? If there is still no way to specify apiVersion - conflict is still here. You just change overriding order (my custom mapping will have bigger priority then deffault)?

manusa · 2023-07-17T08:56:03Z

So i do not undestand what should i test? If there is still no way to specify apiVersion - conflict is still here. You just change overriding order (my custom mapping will have bigger priority then deffault)?

From the issue description it was unclear you were actually requesting to include apiVersion as part of the custom mappings configuration. It is much clearer now (currently working on that, will probably be available on tomorrow's SNAPSHOTs -I'll keep this issue posted-).

What's fixed right now is the possibility to use your Mappings to override the default ones (yes priority override). And the overall way resources are applied. The previous -cr.yml suffix was a hack and not an actual feature.

manusa · 2023-07-17T13:41:25Z

Hi @tipame
With the latest PR everything you mentioned should be covered. You should be able to test this once the nightly build SNAPSHOTs are published.

manusa · 2023-08-01T10:43:26Z

Closing this issue as it should be already fixed. Please re-open or create a new one in case there are problems remaining.

tipame added the bug Something isn't working label Jun 20, 2023

manusa added this to the 1.14.0 milestone Jun 20, 2023

manusa self-assigned this Jun 21, 2023

This was referenced Jun 21, 2023

fix: remove unused ResourceConfig.customResourceDefinitions field #2230

Merged

refactor: split KubernetesResourceUtil + added coverage #2234

Merged

This was referenced Jul 3, 2023

fix: fragment group-version-kind completed only if necessary #2248

Merged

refactor: initial simplification of ApplyService #2250

Merged

This was referenced Jul 11, 2023

test: ApplyService tests prior to further refactoring and method consolidation #2260

Merged

Simplify ApplyService removing redundant applyResource method calls #2277

Merged

manusa added the Waiting on feedback Issues that require feedback from User/Other community members label Jul 12, 2023

manusa mentioned this issue Jul 12, 2023

refactor: remove consolidate redundant applyXxx methods into applyResource #2278

Merged

17 tasks

manusa removed the Waiting on feedback Issues that require feedback from User/Other community members label Jul 17, 2023

manusa mentioned this issue Jul 17, 2023

feat: MappingConfig supports defining apiVersion #2282

Merged

17 tasks

manusa added the Waiting on feedback Issues that require feedback from User/Other community members label Jul 17, 2023

This was referenced Jul 17, 2023

doc: documentation for Kind/Filename mapping with apiVersion #2284

Merged

Revisit and refactor AppyService#applyEntity method #643

Closed

manusa closed this as completed Aug 1, 2023

manusa removed the Waiting on feedback Issues that require feedback from User/Other community members label Aug 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

apiVersion is ignored by resource and apply goals #2219

apiVersion is ignored by resource and apply goals #2219

tipame commented Jun 20, 2023 •

edited

Loading

rohanKanojia commented Jun 20, 2023 •

edited

Loading

tipame commented Jun 20, 2023

rohanKanojia commented Jun 20, 2023

rohanKanojia commented Jun 20, 2023 •

edited

Loading

tipame commented Jun 20, 2023 •

edited

Loading

rohanKanojia commented Jun 20, 2023

rohanKanojia commented Jun 20, 2023

tipame commented Jun 20, 2023

manusa commented Jun 20, 2023

tipame commented Jun 20, 2023 •

edited by manusa

Loading

rohanKanojia commented Jun 20, 2023

manusa commented Jun 20, 2023

tipame commented Jun 20, 2023 •

edited

Loading

rohanKanojia commented Jun 20, 2023 •

edited

Loading

tipame commented Jun 20, 2023

manusa commented Jun 20, 2023

manusa commented Jul 12, 2023 •

edited

Loading

tipame commented Jul 17, 2023

rohanKanojia commented Jul 17, 2023 •

edited

Loading

tipame commented Jul 17, 2023

manusa commented Jul 17, 2023

tipame commented Jul 17, 2023

manusa commented Jul 17, 2023

manusa commented Jul 17, 2023

manusa commented Aug 1, 2023

apiVersion is ignored by resource and apply goals #2219

apiVersion is ignored by resource and apply goals #2219

Comments

tipame commented Jun 20, 2023 • edited Loading

Describe the bug

Eclipse JKube version

Component

Apache Maven version

Gradle version

Steps to reproduce

Expected behavior

Runtime

Kubernetes API Server version

Environment

Eclipse JKube Logs

Sample Reproducer Project

Additional context

rohanKanojia commented Jun 20, 2023 • edited Loading

tipame commented Jun 20, 2023

rohanKanojia commented Jun 20, 2023

rohanKanojia commented Jun 20, 2023 • edited Loading

tipame commented Jun 20, 2023 • edited Loading

rohanKanojia commented Jun 20, 2023

rohanKanojia commented Jun 20, 2023

tipame commented Jun 20, 2023

manusa commented Jun 20, 2023

tipame commented Jun 20, 2023 • edited by manusa Loading

rohanKanojia commented Jun 20, 2023

manusa commented Jun 20, 2023

tipame commented Jun 20, 2023 • edited Loading

rohanKanojia commented Jun 20, 2023 • edited Loading

tipame commented Jun 20, 2023

manusa commented Jun 20, 2023

manusa commented Jul 12, 2023 • edited Loading

tipame commented Jul 17, 2023

rohanKanojia commented Jul 17, 2023 • edited Loading

tipame commented Jul 17, 2023

manusa commented Jul 17, 2023

tipame commented Jul 17, 2023

manusa commented Jul 17, 2023

manusa commented Jul 17, 2023

manusa commented Aug 1, 2023

tipame commented Jun 20, 2023 •

edited

Loading

rohanKanojia commented Jun 20, 2023 •

edited

Loading

rohanKanojia commented Jun 20, 2023 •

edited

Loading

tipame commented Jun 20, 2023 •

edited

Loading

tipame commented Jun 20, 2023 •

edited by manusa

Loading

tipame commented Jun 20, 2023 •

edited

Loading

rohanKanojia commented Jun 20, 2023 •

edited

Loading

manusa commented Jul 12, 2023 •

edited

Loading

rohanKanojia commented Jul 17, 2023 •

edited

Loading