Make legacy resource ordering customizable in the kustomization.yaml file #3913

yanniszark · 2021-05-21T12:44:45Z

Is your feature request related to a problem? Please describe.

Related issue: #3794

TLDR:

Kustomize changed the order it prints its resources.
The new order broke Kubeflow and we can't upgrade to v4+ (Upgrade to use Kustomize v4+ kubeflow/manifests#1797 (comment))
Order can't be reversed because it's been in effect for 1 year now, so changing it will break users.
Instead, we can make the ordering configurable via the kustomization.yaml file, as described in Kustomize resource ordering regression #3794 (comment)

Describe the solution you'd like

Make resource ordering configurable via the kustomization.yaml file.
The sorter accepts the orderFirst and orderLast arguments, to know which resources to order first and last.
Then we need to decide what to call the new field for the sorter in the kustomization.yaml file.

cc @monopole

The text was updated successfully, but these errors were encountered:

KnVerey · 2021-05-25T23:52:59Z

I think that this new field should replace the --reorder flag. Because it changes the output, this flag is currently violating Kustomize's principle of having no build-time side-effects. IMO we should add a deprecation warning to the flag when this new field merges, and then remove the flag from a future major version release. While both are present, the field should take precedence (if you've switched your kustomizations to the new way, we should ignore the old)... though this might be complicated in practice because the flag's scope is global and the field's should not be.

Maybe something like this?

sortOrder: {legacy,fifo} # expandable e.g. if we want to add a non-legacy option that provides semantically stable ordering but without the release order implications
legacySortOptions: #  it's an error if this is present and sortOrder isn't legacy
  orderFirst:
  - {GVK}
  orderLast:
  - {GVK}

Or just the latter field, and stock legacy ordering would be enabled with legacySortOptions: {}. I don't love the clutter of two new top-level fields, but the latter reads awkwardly to me.

@yanniszark wdyt of the above? Are you interested in contributing this field?

yanniszark · 2021-05-28T09:37:01Z

Hi @KnVerey,
I think the discussion about kustomize removing the reorder flag is very interesting and something we should consider after this feature lands. We (Arrikto, Kubeflow) are interested in pushing this feature forward, so I will start with the example you suggested as my goal.

KnVerey · 2021-05-28T15:06:27Z

/assign @yanniszark

numbsafari · 2021-06-25T00:52:55Z

Brought here by some problems deploying cert-manager together with some other tools using kustomize. I think this idea of having a logical default ordering, but the ability on a per-kustomization basis to describe a desired ordering based on resource type makes a lot of sense. The main reason I like it is that you can memorialize/document the need for the ordering within your data/files, rather than relying on an external side-effect registered via a CLI flag. In addition, given the existence of CustomResourceDefinitions with their own potential semantics, it's impossible for kustomize to know "everything" it needs to know.

Extend the kustomization.yaml API with the 'sortOrder' and 'legacySortOptions' fields. Refs kubernetes-sigs#3913 Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>

natasha41575 · 2021-07-07T21:45:04Z

I don't love the clutter of two new top-level fields, but the latter reads awkwardly to me.

Would it make sense to consolidate it into a single top-level field with nested subfields?

Instead of erroring if legacySortOptions is present and sortOrder isn't legacy, we could choose to make sortOrder and legacySortOptions mutually exclusive; the presence of legacySortOptions could implicitly mean that the sort order should be legacy.

To that end, we could just call it legacy instead of legacySortOptions to be more concise.

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- ../base

sortOptions:
  legacy:
    orderFirst:
    - {GVK}
    orderLast:
    - {GVK}

vs

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- ../base

sortOptions:
  sortOrder: fifo

Stock legacy ordering would be enabled with

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- ../base

sortOptions:
  sortOrder: legacy

@KnVerey @yanniszark WDYT?

yanniszark · 2021-07-08T14:43:11Z

@natasha41575 IIUC, you are proposing this:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- ../base

sortOptions:
  legacy:
    orderFirst:
    - {GVK}
    orderLast:
    - {GVK}

VS

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- ../base

sortOptions:
  sortOrder: legacy
  legacySortOptions:
    orderFirst:
    - {GVK}
    orderLast:
    - {GVK}

TBF, the second one seems more intuitive to me. @KnVerey what do you think?

natasha41575 · 2021-07-13T21:27:23Z

@KnVerey please let us know your thoughts

I'm happy with the latter of

sortOptions:
  sortOrder: legacy
  legacySortOptions:
    orderFirst:
    - {GVK}
    orderLast:
    - {GVK}

It was mostly having two new top-level fields that seemed a bit cluttered to me.

yanniszark · 2021-07-21T16:36:38Z

ping @KnVerey
@natasha41575 I will start the implementation of the latter in the PR, while we wait.
Hopefully, there will only be small changes on top of that :)

KnVerey · 2021-07-21T17:08:05Z

That LGTM. I'd suggest dropping sortOrder to just order since it is under sortOptions, but I don't feel strongly about it.

natasha41575 · 2021-07-21T18:09:14Z

I'd suggest dropping sortOrder to just order since it is under sortOptions, but I don't feel strongly about it.

+1

Make orderFirst and orderLast public in preparation of changes to make the resource ordering configurable. Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> kustomization: Extend with sortOrder and legacySortOptions fields Extend the kustomization.yaml API with the 'sortOrder' and 'legacySortOptions' fields. Refs kubernetes-sigs#3913 Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> plugins: Extend LegacyOrderTransformer with customizable ordering Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> plugins: Generate new LegacyOrderTransformer Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> Fix existing LegacyOrderTransformer invocations Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> Throw an error if legacySortOptions is set and sortOrder is not legacy Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> api: Updated dep files after running tests Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> cmd/config: Updated dep files after running tests Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> cmd/pluginator: Updated dep files after running tests Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> kustomize: Updated dep files after running tests Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> wip Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>

k8s-triage-robot · 2021-10-19T18:43:18Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

natasha41575 · 2021-10-19T19:12:47Z

/remove-lifecycle stale

k8s-triage-robot · 2022-01-17T19:52:35Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

KnVerey · 2022-01-17T20:51:32Z

#4019 is back in active development and getting really close now

/remove-lifecycle stale

k8s-triage-robot · 2022-04-17T21:08:43Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot · 2022-05-17T21:45:43Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

bjw-s · 2022-05-24T07:47:41Z

/remove-lifecycle rotten

k8s-triage-robot · 2022-08-22T08:13:07Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

onedr0p · 2022-08-22T09:57:55Z

/remove-lifecycle stale

Kustomize has a new field called "sortOptions" to sort resources: kubernetes-sigs/kustomize#3913 kubernetes-sigs/kustomize#4019 Document what it does and how to use it. Signed-off-by: Yannis Zarkadas <yanniszark@gmail.com>

monopole added triage/accepted Indicates an issue or PR is ready to be actively worked on. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. labels May 25, 2021

monopole assigned KnVerey May 25, 2021

KnVerey removed their assignment May 25, 2021

k8s-ci-robot assigned yanniszark May 28, 2021

KnVerey mentioned this issue Jun 2, 2021

Deprecate the reorder flag in favour of a Kustomization field #3947

Closed

KnVerey added this to To do in Kustomization v1 Jun 9, 2021

This was referenced Jun 15, 2021

Kustomize resource ordering regression #3794

Closed

Upgrade to use Kustomize v4+ kubeflow/manifests#1797

Closed

yanniszark mentioned this issue Jun 30, 2021

Make ordering configurable #4019

Merged

natasha41575 mentioned this issue Jul 7, 2021

Option to retain resource object origin data as a label #3979

Closed

KnVerey mentioned this issue Aug 4, 2021

[Question] instantiation of resource(s) and CRD Kind of resource #3502

Closed

KnVerey mentioned this issue Sep 29, 2021

IngressClass should be ordered before Ingress #4191

Closed

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 19, 2021

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 19, 2021

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 17, 2022

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 17, 2022

vyankyGH mentioned this issue Mar 24, 2022

[Bug] Mutation Rule removes extra properties from Pod manifest kyverno/kyverno#3331

Closed

2 tasks

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 17, 2022

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels May 17, 2022

k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label May 24, 2022

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Aug 22, 2022

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Aug 22, 2022

k8s-ci-robot closed this as completed in #4019 Dec 2, 2022

Kustomization v1 automation moved this from To do to Done Dec 2, 2022

yanniszark mentioned this issue Dec 9, 2022

Document sortOptions field kubernetes-sigs/cli-experimental#306

Merged

remkolems mentioned this issue May 29, 2024

Kustomization json schema - missing sortOptions SchemaStore/schemastore#3825

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make legacy resource ordering customizable in the kustomization.yaml file #3913

Make legacy resource ordering customizable in the kustomization.yaml file #3913

yanniszark commented May 21, 2021

KnVerey commented May 25, 2021

yanniszark commented May 28, 2021

KnVerey commented May 28, 2021

numbsafari commented Jun 25, 2021

natasha41575 commented Jul 7, 2021 •

edited

yanniszark commented Jul 8, 2021

natasha41575 commented Jul 13, 2021 •

edited

yanniszark commented Jul 21, 2021

KnVerey commented Jul 21, 2021

natasha41575 commented Jul 21, 2021 •

edited

k8s-triage-robot commented Oct 19, 2021

natasha41575 commented Oct 19, 2021

k8s-triage-robot commented Jan 17, 2022

KnVerey commented Jan 17, 2022

k8s-triage-robot commented Apr 17, 2022

k8s-triage-robot commented May 17, 2022

bjw-s commented May 24, 2022

k8s-triage-robot commented Aug 22, 2022

onedr0p commented Aug 22, 2022

Make legacy resource ordering customizable in the kustomization.yaml file #3913

Make legacy resource ordering customizable in the kustomization.yaml file #3913

Comments

yanniszark commented May 21, 2021

KnVerey commented May 25, 2021

yanniszark commented May 28, 2021

KnVerey commented May 28, 2021

numbsafari commented Jun 25, 2021

natasha41575 commented Jul 7, 2021 • edited

yanniszark commented Jul 8, 2021

natasha41575 commented Jul 13, 2021 • edited

yanniszark commented Jul 21, 2021

KnVerey commented Jul 21, 2021

natasha41575 commented Jul 21, 2021 • edited

k8s-triage-robot commented Oct 19, 2021

natasha41575 commented Oct 19, 2021

k8s-triage-robot commented Jan 17, 2022

KnVerey commented Jan 17, 2022

k8s-triage-robot commented Apr 17, 2022

k8s-triage-robot commented May 17, 2022

bjw-s commented May 24, 2022

k8s-triage-robot commented Aug 22, 2022

onedr0p commented Aug 22, 2022

natasha41575 commented Jul 7, 2021 •

edited

natasha41575 commented Jul 13, 2021 •

edited

natasha41575 commented Jul 21, 2021 •

edited