Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maintain order of Documents within the global Document Set throughout processing #257

Open
gcheadle-vmware opened this issue Dec 16, 2020 · 1 comment
Labels
bug This issue describes a defect or unexpected behavior carvel accepted This issue should be considered for future work and that the triage process has been completed

Comments

@gcheadle-vmware
Copy link
Contributor

gcheadle-vmware commented Dec 16, 2020

What steps did you take:
tests in the tkg-cli-provider repo CI

What happened:
The list of line numbers is non-deterministic, failure from the CI shows:

-Error: : unable to get template: Overlaying (in following order: overlay.yaml, aws-overlay.yaml, 01_plans/prod.yaml, 
03_customizations/autoscaler/autoscaler_overlay.yaml, 03_customizations/http_proxy.yaml): Document on line 
03_customizations/http_proxy.yaml:62: Expected number of matched nodes to be 1, but was 3 (lines: base-
template.yaml:125, aws-overlay.yaml:150, aws-overlay.yaml:163)
+Error: : unable to get template: Overlaying (in following order: overlay.yaml, aws-overlay.yaml, 01_plans/prod.yaml, 
03_customizations/autoscaler/autoscaler_overlay.yaml, 03_customizations/http_proxy.yaml): Document on line 
03_customizations/http_proxy.yaml:62: Expected number of matched nodes to be 1, but was 3 (lines: aws-overlay.yaml:150, 
aws-overlay.yaml:163, base-template.yaml:125)

What did you expect:
Expected (lines: base-template.yaml:125, aws-overlay.yaml:150, aws-overlay.yaml:163) to be in the same order for every run.

Anything else you would like to add:
Accepted #256 as a temporary fix until the Carvel team can determine and fix the upstream source of non-determinism.
Slack link with further context.

Environment:

  • ytt version (use ytt --version):
  • OS (e.g. from /etc/os-release):
@gcheadle-vmware gcheadle-vmware added bug This issue describes a defect or unexpected behavior carvel triage This issue has not yet been triaged for relevance labels Dec 16, 2020
@cari-lynn cari-lynn added carvel accepted This issue should be considered for future work and that the triage process has been completed and removed carvel triage This issue has not yet been triaged for relevance labels Dec 16, 2020
@pivotaljohn
Copy link
Contributor

@gcheadle-vmware and I reviewed this together:

  • consider whether this solution is "sufficient" — sorting the output directly, while addresses the symptom, leaves open the possibility that any future output relying on the order of documents in the output of templates may accidentally also be non-deterministic.
  • non-determinism comes from iterating over a map — looks like the heart of the non-deterministic behavior comes from how we're storing the total library of YAML Documents: in a map. When document matchers go searching over the total set, we're iterating over that map.
  • be conscious of any performance trade-off — if we do decide to sacrifice some performance for more centralized determinism, please have data to quantify the performance difference (e.g. benchmark a large-ish library before and after).

@pivotaljohn pivotaljohn changed the title Non-deterministic ordering in overlay error messages Maintain order of Documents within the global Document Set throughout processing Nov 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue describes a defect or unexpected behavior carvel accepted This issue should be considered for future work and that the triage process has been completed
Projects
Status: To Triage
Development

No branches or pull requests

3 participants